Rajeev’s Weblog

Chatting with webmasters often reveals widespread beliefs that might have been accurate in the past, but are not necessarily up-to-date any more. This was the case when we recently talked to a couple of friends about the structure of a URL. One friend was concerned about using dynamic URLs, since (as she told us) “search engines can’t cope with these.” Another friend thought that dynamic URLs weren’t a problem at all for search engines and that these issues were a thing of the past. One even admitted that he never understood the fuss about dynamic URLs in comparison to static URLs. For us, that was the moment we decided to read up on the topic of dynamic and static URLs. First, let’s clarify what we’re talking about:

What is a static URL?
A static URL is one that does not change, so it typically does not contain any url parameters. It can look like this: http://www.example.com/archive/january.htm. You can search for static URLs on Google by typing filetype:htm in the search field. Updating these kinds of pages can be time consuming, especially if the amount of information grows quickly, since every single page has to be hard-coded. This is why webmasters who deal with large, frequently updated sites like online shops, forum communities, blogs or content management systems may use dynamic URLs.

What is a dynamic URL?
If the content of a site is stored in a database and pulled for display on pages on demand, dynamic URLs maybe used. In that case the site serves basically as a template for the content. Usually, a dynamic URL would look something like this: http://code.google.com/p/google-checkout-php-sample-code/issues/detail?id=31. You can spot dynamic URLs by looking for characters like: ? = &. Dynamic URLs have the disadvantage that different URLs can have the same content. So different users might link to URLs with different parameters which have the same content. That’s one reason why webmasters sometimes want to rewrite their URLs to static ones.

Should I try to make my dynamic URLs look static?
Following are some key points you should keep in mind while dealing with dynamic URLs:
1.It’s quite hard to correctly create and maintain rewrites that change dynamic URLs to static-looking URLs.
2.It’s much safer to serve us the original dynamic URL and let us handle the problem of detecting and avoiding problematic parameters.
3.If you want to rewrite your URL, please remove unnecessary parameters while maintaining a dynamic-looking URL.
4.If you want to serve a static URL instead of a dynamic URL you should create a static equivalent of your content.

Which can Googlebot read better, static or dynamic URLs?
We’ve come across many webmasters who, like our friend, believed that static or static-looking URLs were an advantage for indexing and ranking their sites. This is based on the presumption that search engines have issues with crawling and analyzing URLs that include session IDs or source trackers. However, as a matter of fact, we at Google have made some progress in both areas. While static URLs might have a slight advantage in terms of clickthrough rates because users can easily read the urls, the decision to use database-driven websites does not imply a significant disadvantage in terms of indexing and ranking. Providing search engines with dynamic URLs should be favored over hiding parameters to make them look static.

Let’s now look at some of the widespread beliefs concerning dynamic URLs and correct some of the assumptions which spook webmasters.

Myth: “Dynamic URLs cannot be crawled.”
Fact: We can crawl dynamic URLs and interpret the different parameters. We might have problems crawling and ranking your dynamic URLs if you try to make your urls look static and in the process hide parameters which offer the Googlebot valuable information. One recommendation is to avoid reformatting a dynamic URL to make it look static. It’s always advisable to use static content with static URLs as much as possible, but in cases where you decide to use dynamic content, you should give us the possibility to analyze your URL structure and not remove information by hiding parameters and making them look static.

Myth: “Dynamic URLs are okay if you use fewer than three parameters.”
Fact: There is no limit on the number of parameters, but a good rule of thumb would be to keep your URLs short (this applies to all URLs, whether static or dynamic). You may be able to remove some parameters which aren’t essential for Googlebot and offer your users a nice looking dynamic URL. If you are not able to figure out which parameters to remove, we’d advise you to serve us all the parameters in your dynamic URL and our system will figure out which ones do not matter. Hiding your parameters keeps us from analyzing your URLs properly and we won’t be able to recognize the parameters as such, which could cause a loss of valuable information.

Following are some questions we thought you might have at this point.

Does that mean I should avoid rewriting dynamic URLs at all?
That’s our recommendation, unless your rewrites are limited to removing unnecessary parameters, or you are very diligent in removing all parameters that could cause problems. If you transform your dynamic URL to make it look static you should be aware that we might not be able to interpret the information correctly in all cases. If you want to serve a static equivalent of your site, you might want to consider transforming the underlying content by serving a replacement which is truly static. One example would be to generate files for all the paths and make them accessible somewhere on your site. However, if you’re using URL rewriting (rather than making a copy of the content) to produce static-looking URLs from a dynamic site, you could be doing harm rather than good. Feel free to serve us your standard dynamic URL and we will automatically find the parameters which are unnecessary.

Can you give me an example?
If you have a dynamic URL which is in the standard format like foo?key1=value&key2=value2 we recommend that you leave the url unchanged, and Google will determine which parameters can be removed; or you could remove uncessary parameters for your users. Be careful that you only remove parameters which do not matter. Here’s an example of a URL with a couple of parameters:

www.example.com/article/bin/answer.foo?language=en&answer=3&sid=98971298178906&query=URL
language=en – indicates the language of the article
answer=3 – the article has the number 3
sid=8971298178906 – the session ID number is 8971298178906
query=URL – the query with which the article was found is [URL]
Not all of these parameters offer additional information. So rewriting the URL to www.example.com/article/bin/answer.foo?language=en&answer=3 probably would not cause any problems as all irrelevant parameters are removed.

The following are some examples of static-looking URLs which may cause more crawling problems than serving the dynamic URL without rewriting:
www.example.com/article/bin/answer.foo/en/3/98971298178906/URL
www.example.com/article/bin/answer.foo/language=en/answer=3/sid=98971298178906/query=URL
www.example.com/article/bin/answer.foo/language/en/answer/3/
sid/98971298178906/query/URL
www.example.com/article/bin/answer.foo/en,3,98971298178906,URL

Rewriting your dynamic URL to one of these examples could cause us to crawl the same piece of content needlessly via many different URLs with varying values for session IDs (sid) and query. These forms make it difficult for us to understand that URL and 98971298178906 have nothing to do with the actual content which is returned via this URL. However, here’s an example of a rewrite where all irrelevant parameters have been removed:
www.example.com/article/bin/answer.foo/en/3
Although we are able to process this URL correctly, we would still discourage you from using this rewrite as it is hard to maintain and needs to be updated as soon as a new parameter is added to the original dynamic URL. Failure to do this would again result in a static looking URL which is hiding parameters. So the best solution is often to keep your dynamic URLs as they are. Or, if you remove irrelevant parameters, bear in mind to leave the URL dynamic as the above example of a rewritten URL shows:
www.example.com/article/bin/answer.foo?language=en&answer=3

Just 48 hours into the launch of Google’s web browser, Google Chrome, Mozilla Inc, creators of the famed and widely used FireFox web browser are already feeling the heat. In lieu of the rising popularity of Google Chrome, Mozilla has begun taking damage control steps.

According to TechCrunch, Google Chrome has been created on the V8 platform, that happens to be the fastest and the most reliable browser engine in the world, making Chrome operations a breeze. Since, last two days, based on my extensive usage of Google web browser, I would certainly agree to this fact.

However, Mozilla has now announced that by the end of the year 2008, it will incorporate, TraceMonkey, that will enhance Firefox’s performance at incredible levels, hence making it the fastest browser on the web. This new engine will be rolled out in the upcoming release of FireFox 3.1.

As per TechCrunch, “To prove it, Mozilla tested Firefox running on TraceMonkey and compared it to Google’s Chrome beta using its own benchmarking solution called SunSpider. According to the company, Chrome was 28 percent slower on Windows XP and 16 percent slower on Windows Vista.
Mozilla is quick to point out that TraceMonkey has only been in development for a few months and will only get better before it’s rolled out later this year, but the company has a vested interest in seeing Firefox come out on top in its benchmark testing, so all figures should probably be questioned, to say the least.”

After much speculations and waiting, Google has finally announced the launch of their own web browser, known as ‘Google Chrome’. This all new multi-functional web browser will be available for download in over 100 countries on Tuesday, September 2nd 2008. However, as with every other Google service, Google Chrome will be launched in the BETA mode. As of now, Google Chrome is exclusively available for Microsoft Operating System platforms, but Google insists it will be releasing versions of the same for Mac and Linux too. The Google browser has been integrated with components from Apple’s WebKit and Mozilla’s Firefox. In lieu of the various components involved in the creation of Google Chrome, Google has decided to keep it as a open source browser.

The introduction of Google Chrome has been rather dramatic, with Google introducing its all new open source browser, via the means of a comic book. You find the Google Comic Book, here.

Google Chrome is known to have the following features:

1. An open source platform including Google Gears integration.

2. JavaScript Virtual Machine called V8.

3. Special tabbed browsing functionality.

4. Autocompleting & search powered address bar.

5. Incognito mode where “nothing that occurs in that window is ever logged on your computer”

6. Ability to launch web applications in browser windows.

7. Advanced privacy / malware / phishing protection.

According to Google, “On the surface, we designed a browser window that is streamlined and simple. To most people, it isn’t the browser that matters. It’s only a tool to run the important stuff — the pages, sites and applications that make up the web. Like the classic Google homepage, Google Chrome is clean and fast. It gets out of your way and gets you where you want to go. Under the hood, we were able to build the foundation of a browser that runs today’s complex web applications much better. By keeping each tab in an isolated “sandbox”, we were able to prevent one tab from crashing another and provide improved protection from rogue sites. We improved speed and responsiveness across the board. We also built a more powerful JavaScript engine, V8, to power the next generation of web applications that aren’t even possible in today’s browsers.”

Screen Shots of Google Chrome:

Google Chrome from the service’s frontpage

The auto-completion of the so-called “omnibox” address bar

The homepage showing 9 thumbnailed pages to access, along with more pointers in the side-bar, to appear “[e]very time you open a new tab”, as Google says.

This screenshot shows Google Calendar and a dialog reading “Create shortcuts in the following locations”, listing Desktop, Start Menu and Quick Launch Bar.

Zooming in on the browser tabs.

The Google Chrome task manager, e.g. to monitor if certain sites cause memory problems.

A screen showing the “Google incognito” mode for allegedly more private browsing.

Another auto-completion example.

A star near the address input bar lets you bookmark a page, apparently.

A look into the settings menu.

Google in their tour says with Chrome “you see your download’s status at the bottom of your current window.”

Here is a demo video of Google Chrome:

Mozilla Inc. that is the inventor of the widely acclaimed web browser, FireFox has announced that it has extended its search deal with the search engine giant, Google for another three years. According to the negotiations between Google and Mozilla, in exchange of setting Google as the default search engine, Google will pay Mozilla a hnadsome amount of $57 million amounting to a total of 85% of Mozilla’s annual revenue. The new deal will run till throughout the year 2011.

According to Mitchell Baker (Mozilla, CEO), “We’re very, very happy about our relationship with Google and this makes sure that Mozilla will be sustainable and thrive for quite a long time to come. We develop our product and technical direction as part of an open process unrelated to the search relationship with Google. We talk to Google about the parts of the product that offer Google services (i.e., the Firefox Start Page) and the services they provide, like anti-phishing. Otherwise Google does not have any special relationship to Mozilla project activities.

The monetary benefits provided to Mozilla due to search deal, enables the company to pay staff, support its bandwidth and hardware infrastructure, and to distribute a number of grants.