The Analyst

Why aren’t Yahoo and Bing crawling your site despite submitting sitemaps and URLs

The Analyst — Thu, 25 Feb 2010 08:56:00 +0000

Google, Yahoo, Bing, Baidu, Yandex all have their own way of crawling web pages. But I should say that Yahoo and Bing are very big misers.

Google crawls a site with the speed corresponding to the frequency and quantity of updates of that site, till today Googlebot is the best crawler there has been (among search engines), they will add your new site to their index in no time if it's worth adding and displaying in search results, though they are the best but they are still slow (it is to me, will be to you too if you have a bigger site running overnight).Though google considers link popularity in ranking webpages for search results, it almost doesn't consider link popularity of a site for crawling it

Baidu and Yandex visit sites periodically, not sure if they consider the updating speed of site or the link popularity.

If I have to mention a single reason why Yahoo and Bing have lost their war against Google, it's really really because of their incapable crawlers. Both of them consider link popularity, domain age, domain authority and of course the size of the site for crawling it, it makes them slower and because of the millions of new webpages showing up everyday they will always be behind Google in time.

Yahoo is considerably better than Bing in Crawling, Yahoo has crawled one of my site (which doesn't have external backlinks) almost completely in one year, bing has listed only the main page yet

Google crawl rate levels in google webmaster tools

The Analyst — Wed, 17 Feb 2010 14:34:00 +0000

By default Google sets your site's crawl rate according to the frequency of new pages that you create, you can change the speed up to a maximum of 0.5 requests per second (i.e 2 seconds delay between requests) inside webmaster tools at Dashboard >> Site configuration >> settings, as your site grows you can increase the crawl rates beyond the previously maximum 0.5 requests per second.

Your site's crawl rate doesn't randomly increase, it increases in levels
#1 Maximum of 0.5 requests per second
#2 0.75 requests per second
#3 1 request per second and more

Your site's crawling speed may jump to any of these levels depending on the speed of growth of your site, please mention your site's crawling speed if it's more than 1 request per second

What does Dreamworks SKG stands for ?

The Analyst — Wed, 17 Feb 2010 13:25:00 +0000

SKG in Dreamworks SKG stands for "Spielberg (Steven Spielberg), Katzenberg (Jeffrey Katzenberg) and Geffen (David Geffen)", the initials of second names of the three producers who united to create a big film studio called Dreamworks, frankly I have no clear idea what that means, does that mean dream works! or works of dream

How to search for a string in the URL of a site’s pages

The Analyst — Wed, 17 Feb 2010 04:10:00 +0000

In order to search if a site has a specific key phrase (that we are looking for) anywhere inside it’s pages we know site: operator could work.

I have a site which has pages translated into other languages, and the url structure for other languages is like

http://mysite.com/page/1/zhs/ and http://mysite.com/page/1/zht/ I can’t use site:http://example.com/zhs/ to check how many pages from the mini site http://mysite/zhs/ are added to Google’s index, I use site:http://example.com/ inurl:/zhs/ and it only lists urls containing /zhs/

This inurl: search operator associated with site: operator is very useful not just to check pages indexed in google but also to search within pages of sites in which title of the page is a part of the page’s URL, wikipedia.org, answers.com, about.com and even blogspot.com blogs are good examples where you can use this.

For Example take this query site:blogspot.com/p/ inurl:about it lists about me, about us, about this site kind of pages from blogspot.com pages

Using substr() php function with foreign languages

The Analyst — Sat, 13 Feb 2010 15:03:00 +0000

Chinese, Thai, Japanese, Korean, Arabic, Hebrew and other non-latin characters are of more than one character length in real terms.
Since an alphabet takes more than one byte it is not considered as single character by PHP engine, thus it chops off some characters sometimes, thus operating on text in these languages with SUBSTR function is not recommended. Thus

However there is another function mb_substr() it only cuts the string after a proper end of character.
Eg usage : $final_string=mb_substr($initial_string,0,50) instead of $final_string=substr($initial_string,0,50)

Donate to Haiti earthquake relief from you adsense account

The Analyst — Sat, 13 Feb 2010 14:48:00 +0000

If you had no credit card and no other means of helping Haiti people but an adsense account you have a chance to help them, Check AdSense Donations for Haiti Earthquake Relief, that is very kind of google, they always come up with ideas and heart to help people

Mininova’s attempt to reduce piracy and at’s impact on it

The Analyst — Sat, 13 Feb 2010 14:42:00 +0000

Mininova the giant torrent portal has taken steps to reduce piracy by deciding to share only the copy righted torrent materials. Number of visitors has come down suddenly It’s alexa rank has come down from around 100 to 1000, that’s infinite in terms of website traffic.

They are probably gonna lose all that scope they had because of their immediate step to do the right

Why is blogger preferable for blogging?

The Analyst — Sat, 13 Feb 2010 09:53:00 +0000

Firstly it’s free, you don’t pay a rusted coin, neither for hosting nor for domain name, of course you have to be happy with the sub domain.and
secondly it’s because I suspect Google favors BlogSpot (no solid proof though, I feel happy to think it that way)
and thirdly because it’s better to have guys better than you do your work for you. I mean it’s bit risky to have your own Worpress blog and managing it for yourself (updating it often, updating plugins (everything is automized now), worrying about what if queries exceed, managing broken things, Database, Hackers packers, remembering another password, installing WP, woes continue) (Guys, having self hosted WordPress blog is really worth it, Infact I have some WP blogs myself, but it’s bit risky as I said)
and you can be sure Google doesn’t hire fools to manage your work

List of sites to follow cricket online, those who provide live scores and commentery

The Analyst — Sat, 13 Feb 2010 09:37:00 +0000

Cricinfo: is the best of all sites I have used so far, It covers all international cricketing events, news and more. Many former cricketrs, experts and players write their blogs in it and it makes cricinfo very resourceful, what I love most about them is their statistics and its management, They have a large database of previous matches and stats, I mean it’s seriously difficult to have count of all balls that Sachin and Dravid have played in test matches (They have ball to ball stats of most of the players, except few (Sachin?) of corse), BTW did you know that Afridi scored the fastest century in all forms of cricket on the very first match he had a chance to bat
Cricket Nirvana: I came to know of this site through TV advertisements, I was curious to know what all that fuss was about, It’s actually a good one and they are putting in lot of efferts to make it good
Sify sports: Sify is another good site, I use it often along with cricinfo for live scores, I haven’t really checked if they have any previous matches database
Cricbuzz, Yahoo Cricket are also good ones

Can we watch live cricket matches online without paying money?

The Analyst — Sat, 13 Feb 2010 09:20:00 +0000

The answer is yes! we can watch live cricket online, but mostly it’s illegal. There are plenty of sites which telecast live cricket matches, you can find some by googling the phrase watch cricket online.

Tensports telecasts live matches that it has rights for on its website for free and it’s legal, you can watch cricket from anywhere through tensports.

ESPN also telecasts matches live through internet but only in United States and for those whose ISPs have something to do with ESPN. Sky sports also publishes live matches but it’s restricted only to United Kingdom, may be you can try your luck through PROXY servers hosted in those countries, Funny world!