Google Description Screwed
2015-02-17, 05:00 PM,
#1
Anyone seen this error before?

[Image: image.png]

This is my robots.txt

User-agent: *
Disallow:

Twice this has happened now, not sure exactly why but it would be interesting too know.
Reply
2015-02-17, 05:01 PM, (This post was last modified: 2015-02-17, 05:10 PM by BlackHatSEO.)
#2
You blocked every search engine with your robots.txt because this kind of "Disallow" could be interpreted wrong.

Better delete your robots.txt completely in this case (no further commands) or use a

Code:
Allow: /

instead.

One of the best (and oldest) tutorials: http://www.robotstxt.org/robotstxt.html

This place is intentionally left blank.
Reply
2015-02-17, 05:06 PM,
#3
(2015-02-17, 05:01 PM)BlackHatSEO Wrote: You blocked every search engine with your robots.txt.

One of the best (and oldest) tutorials: /robotstxt.html

i have this:

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://www.mysite.com/atom.xml?redirect=...esults=500

is it good?
i have blogger blog.
Reply
2015-02-17, 05:10 PM,
#4
(2015-02-17, 05:06 PM)darkjosephx90 Wrote: i have this:

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: /atom.xml?redirect=false&start-index=1&max-results=500

A

Code:
Disallow:
(without any parameters)

could always be interpreted wrong, because some spiders can't handle it.

Better use this instead:

Code:
Allow: /
Latest Thanks - View all

Mr White(2015-02-18 04:09 AM) darkjosephx90(2015-02-17 05:14 PM) 

This place is intentionally left blank.
Reply
2015-02-17, 05:12 PM, (This post was last modified: 2015-02-17, 05:13 PM by darkjosephx90.)
#5
so i should do it like this?
User-agent: Mediapartners-Google
Allow: /
User-agent: *
Disallow: /search
Allow: /
Sitemap: /atom.xml?redirect=false&start-index=1&max-results=500

sorry for asking,i'm seo noob o_O
Reply
2015-02-17, 05:17 PM, (This post was last modified: 2015-02-17, 05:17 PM by BlackHatSEO.)
#6
(2015-02-17, 05:12 PM)darkjosephx90 Wrote: so i should do it like this?

If you want to allow the access to your whole site for the Mediapartners bot, and no access to the directory "/search" for all bots, and the full access to your whole site for all bots (except the directory "/search"), then this is correct.

Quote:sorry for asking,i'm seo noob o_O

No problem, you're welcome. I was a SEO noob too, back in 2002 or so. ;-)

This place is intentionally left blank.
Reply
2015-02-17, 05:26 PM, (This post was last modified: 2015-02-17, 05:29 PM by darkjosephx90.)
#7
(2015-02-17, 05:17 PM)BlackHatSEO Wrote:
(2015-02-17, 05:12 PM)darkjosephx90 Wrote: so i should do it like this?

If you want to allow the access to your whole site for the Mediapartners bot, and no access to the directory "/search" for all bots, and the full access to your whole site for all bots (except the directory "/search"), then this is correct.


Quote:sorry for asking,i'm seo noob o_O

No problem, you're welcome. I was a SEO noob too, back in 2002 or so. ;-)

and if i allow all bots to my /search what will happen?
google is my main traffic source and i use that robots.txt code for almost a year,so far it went good,my pages are indexed.
i get number 1 on page 1 on google search for some niches,i also get a lot of help from google plus.
i have a secondary blog.and paste the code to it,see what happens there cause don't wanna screw my main one.
Reply
2015-02-17, 05:52 PM, (This post was last modified: 2015-02-17, 05:52 PM by BlackHatSEO.)
#8
(2015-02-17, 05:26 PM)darkjosephx90 Wrote: and if i allow all bots to my /search what will happen?

This really depends on the content of your "/search" directory. The Google spider is known for having some problems with JavaScript code. Additionally the Google spider likes to get into the "De*p Web" (aka "Invisible Web" or "Hidden Web") [1] by trying to fill out forms and search forms.

So I wouldn't allow access to "/search" and instead stick with your Sitemap.

[1] http://en.wikipedia.org/wiki/Deep_Web
Latest Thanks - View all

darkjosephx90(2015-02-17 06:54 PM) 

This place is intentionally left blank.
Reply
2015-02-17, 06:55 PM,
#9
(2015-02-17, 05:52 PM)BlackHatSEO Wrote:
(2015-02-17, 05:26 PM)darkjosephx90 Wrote: and if i allow all bots to my /search what will happen?

This really depends on the content of your "/search" directory. The Google spider is known for having some problems with JavaScript code. Additionally the Google spider likes to get into the "De*p Web" (aka "Invisible Web" or "Hidden Web") [1] by trying to fill out forms and search forms.

So I wouldn't allow access to "/search" and instead stick with your Sitemap.

[1] /wiki/Deep_Web

okay thanks,i understand now,i'll leave it like this,after all i had good results with this robots.txt and sitemap.
Reply


Possibly Related Threads...
Thread Author Replies Views Last Post
  Google dance martek 7 164 2017-12-16, 05:55 PM
Last Post: CharlieHarper
  Create AdWords account by calling Google kderow 10 180 2017-12-15, 07:19 PM
Last Post: hallow8
  Google deindex martek 4 130 2017-12-13, 01:35 PM
Last Post: martek
  Google Adwords & Tracking software question ArtoAllan 6 198 2017-10-11, 03:00 AM
Last Post: ArtoAllan
  in youtube game videos how to add game image and name in description vinit91 3 185 2017-10-06, 04:23 PM
Last Post: yesman





About Us | Contact Us | CPA Elites | Advertise | Stats | Staff Team

© 2013-2017 CPA Elites Ltd
Enhanced by MyBB and WallBB
Return to top