• Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 838 other followers

  • Karen Kallets Twitter

  • Recent Posts

  • Top Clicks

    • None
  • Pages

  • Top Posts

  • Categories

  • Recent Comments

    Shana on How can you get your blog inde…
    Quentin on Blogging Mistakes for Beg…
    Adell on What other blogs are saying ab…
    Jackie on Ping-O-Matic is your one stop…
    Hermine on What other blogs are saying ab…
    Ans on Tools for creating business we…
    Harris Ferencz on 2010 Marketing Plans: Facebook…
    Kevin Moreland on How can you get your blog inde…
    Twitter on Retweeting your own tweet…
    spam filtering on How can you get your blog inde…
    tweet on Retweeting your own tweet…
    Rich on 5 Best Printer Friendly WordPr…
    rtyecript on How can you get your blog inde…
    cp cheats on How can you get your blog inde…
    cp cheats on Ping-O-Matic is your one stop…
  • Archives

CMO Guide to Digital and Social Media info now posted on LinkedIn with a Twitter Feed

Times change and so does this blog.  About a year ago I switched from posting directly on this blog to feeding my reading and info via LinkedIn and Twitter to this blog.  The info is now fed into this blog for your reading pleasure in the feeds along the sidebar.  The tools are still used heavily as well as the article recaps that have been done over the years.

You can follow directly at Twitter @KarenKallet or follow me on LinkedIn with a requests that indicates you are a follower of my blog “CMO Guide” and with that code, I will add you. LinkedIn.com/KarenKallet.  I usually have three posting per day on things I am reading that you should know about.

Enjoy

Web platforms: Reintroduction of basic website feature functions and benefits

Ebay had an ad on Facebook for EBay fashion that caught my eye.  It caught my eye because back in the day, I used Ebay quite a bit.  Now not at all. 

I  just posted some furniture on Craig’s list and got no response.  I used to get it sold that day. 

Plaxo use to be my digital address book, now I do not even open their emails. 

I don’t even know my MySpace log in, nor have I used it in years.

So what is going on?  I have created a lot of litter on the digital highway with my usage. Self created, but none the less add to a CMO’s marketers dilemma.

My thought is that even the most famous websites or platforms need to reintroduce themselves frequently to their current user base.  The key driver is to reintroduce what they do, how to do it and why they are still relevant.  The basic 101 of the site. 

Most marketers as of late have been focused on organic optimization of the website and content, which is great, but have forgotten the basics of if and when they find you, they better know how to use you and why to use your site.

I still  have preconceived ideas about Ebay based on my last use.  They have may changes dramatically, yet I still shy away from getting reintroduced. Nor has Ebay even tried to re-engage me. Running a sale is different from reintroducing me to how to use the site.

Google Marketplace has miss stepped over and over again.  I was a huge loyalist, then they because so complex that it drove out all small business with having to add ownership tags that drove out most who do not have in-house web techs. Now they are charging per click to make money off the big brands.

 As compared with Amazon market place, which still has some bugs to work out, but which I sing their praises every day.

As the CMO, dig a little deeper with your team. Look at your own website.  Is traffic up or down, not net new traffic, your returning traffic.  Why are they using your site, what are they doing, what are they not doing. Most sites are designed for net new, not returning visitors.

Returning visitors need  their own path of re-education on how to use the site, whats new, what is familiar. Just as same store sales is critical for brick and mortar, it is critical to activate those who are returning.

The digital road is littered with use and disuse.  What state is your website or platform in? What are your returning customers needing?

These simple questions could improve conversion 10% and who would not want that these days.

CardMunch is a CMO’s virtual admin assistant

CardMunch by Linkedin is something all CMO’s need to download on their phones and use. I have stacks of business cards that I want to convert to digital and now can.

It allows you to take a picture of business cards you get, you then send it to them with a push of a button, they transcribe and send it back to you, so you can save to your phone or email or forward it. Better than an admin….

It works well about 80% of the time. Sometimes the person transcribing does not fill in all the info on the card. So you have to send it back.

Once you get enough cards loaded it even alphabetized it for you in a mini address book. It also ties to your Linked in account, so pictures of the person is added.

It is free and worth trying. cardmunch.com or down load from the app store.

What is also nice is you have a photo of the card also.

This allows me to see any notes I may have put on the card.

How can you get your blog indexed by Google in 24 hours?

Getting Google to index your blog posts quickly will help get the traction you want for your blogging efforts.  Many times the editorial staff who writes and posts the blogs, leave out these key steps. Many do not know you need to do this. Some say it now happen automatically.  My advice is to go over each step for your blogs and see where you cam pick up improvements.

The article that I point most to  is one from Quick Online Tips .  “Quickonline tips  Guest post by Andrew Rondeau
“How can you get your blog indexed by Google in 24 hours? There is a lot of conflicting search engine optimization information available, especially when it comes to trying to beat the natural ranking systems and having your blog rank high in Google right from the start.  While it’s possible to make a blog and have it indexed almost straight away, most people go about doing so with a poor overall strategy for success.

Ranking high on Google is all about long-term thinking, and going out to win everything in the first day will often see you wasting time and opportunity without getting anything in return. However, it’s very much possible to be indexed by Google in 24 hours or less. After you make a blog, just follow these simple ideas and you’ll see your website show up in the search results within a day.

1: Submit your blog to Google

 Obviously you’ve got to submit your new blog to Google before it can be indexed. Some free blogging platforms will submit your blog for free, while others, especially self-hosted blogging platforms, require you to do it yourself. Either way, it’s an easy 5-minute job that can be done alongside other, more time-consuming tasks. Use the Google webmasters tools to verify your site. Here’s how:

Step 1: Go to google.com/webmasters. Sign in using your Google Account or create an account if you don’t have one already.

Step 2: Add your blog site by clicking the “Add a site” button. Then type in the domain name you wish to add, and click “Continue”.

Step 3: Once added you will need to verify that you own this site. This can be done one of two ways, Meta Tag or Upload a HTML file. The Meta Tag way is, by far the easiest.

Step 4: Meta Tag verification. This method requires you to add a piece of code to your header.php file. Copy the Meta Tag code Google has asked you to copy to your blog site.

Step 5: In your WordPress dashboard go to “Appearance”, then “Editor” using the menu tabs on the left hand side.

Then click on Header (header.php) on the right hand side to open up the header template. Find the <head> tag highlighted, and BELOW it “paste” in the verification Meta tag Google has given you.

Step 6: Then scroll down and click on “Update File”

Step 7: Go back to the Google Webmasters Tool website and click “Verify” and Google will confirm that you own the site.

Step 8: After verifying, Google will display a confirmation page.

2: Submit your blog Sitemap to Google

If you are using WordPress, install the Google XML sitemap plugin that creates a sitemap of your blog that search engines can read.
You now want to add your blog’s sitemap. This is how:

 Step 1: From within the Google Webmaster tools website, click on the domain name you have just added and it will open up a Dashboard for your site. On the left hand side menu, click on “Site Configuration” and then “Sitemaps”.

Step 2: Click “Submit a Sitemap” and enter the name of your sitemap. It should be sitemap.xml

It will take a few hours for the sitemaps to be analyzed and added. Then you can return here and make sure there are no errors, see how many URLs are being indexed and the time that Google last visited you. Even without the sitemap you will begin to get a list of the top search queries that your site is being found for, incoming links to your site, and the keywords that Google thinks your site is optimized for.

By verifying your site and adding your blog’s sitemap, you are telling Google your blog has arrived and it will quickly be indexed.

3: Use social bookmarking to generate links right away

Social bookmarking services are a great way to generate links to your new blog in record time. From simple community bookmarking websites to the giants of the web, submitting your website to social bookmarking services like StumbleUpon, Digg and Twitter can help you generate powerful, natural linkbacks. Log on and “favorite” your blog, and be sure to encourage your friends and colleagues to do the same.

4: Get posting on forums

Get your blog link in your forum signature and start posting it on as many forums as you can. Of course, spamming is never a good long-term idea, but some smart, relevant forum posts are a great way to bring in new SEO juice and links for your new blog. For some reason, Google tends to prioritize websites that are generating links already, so get out there and create as many as you possibly can for your new blog.

5: Install SEO plugins for your blog

Most blogs will come with a SEO pack available, and WordPress has a free one for download if you’re having trouble optimizing your blog. Search engine optimization is absolutely essential for generating blog traffic, and without putting time and thought into your SEO efforts it’s easy to lose what could be a great opportunity. Even though we’re focusing on short-term SEO speed, it’s good to think long-term right from the beginning. Download and use the All-in-one-Seo-Pack plugin.  Check out my video to optimize your All-in-one-SEO-pack plugin set up: How to use the All in one SEO WordPress plugin to your advantage

Five ideas that take hardly any time will ensure you get you new blog seen and indexed by Google within 24 hours.”

Blogging Mistakes for Beginners

I was researching best practices in blogs the other day and come upon an article that Hubspot did: The Fundamentals of Business Blogging that is worth repeating.

“Seven Easily-Avoided Mistakes for Beginners   1.  Never actually launching it:  This is likely the most common mistake.  Sure, you’ve been meaning to get your blog kicked off.  You may have even authored your first article.  But guess what, it’s not a blog until you actually publish something.   
2.  Using a sub-domain (ex: myblog.typepad.com):  If you’re going to go to the trouble to start a blog, then you owe it to yourself to get started right.  Even if you decide to use one of the many popular free blogging services, you should register your own domain name.  There are many reasons for this, but the most important is control over your website URL.  This is one of the few decisions about your blog that is going to be difficult to fix later.  Trust me on this one, the price is worth it.  [Note to self:  Add a check to the website grader to let people know when they’ve made this blunder] 
3.  Spending too much time on design: Don’t let the design of your blog get you bogged down.  Pick one of the many existing templates out there that are free or close to it and go ahead and get started.  You can always change templates later.  Besides, blogs are fundamentally about content.  Aesthetics help, but your first focus should be on authoring great content.  
4.  Not telling anyone about it: The beginning days of a blog are the hardest.  You don’t have anyone linking to you, the search engines are not ranking you (and maybe not even indexing you).  So, there’s really only one source of traffic:  Direct contact.  The best way to get a new blog launched is to tell people in your network that you now have a blog.  Many people are a little reluctant to do this becomes it seems vain or boastful.  As long as you’re sending the notification to people who have an interest in your topic, there’s nothing wrong with it.  
5.  Getting discouraged too early: Most things in life worth doing take time.  Blogging is no different.  You should give yourself at least 6 months (perhaps even a year) to determine what kind of interest there is out there before giving up due to lack of traffic.  Early momentum is hard, but once things start moving, lots of things will start working in your favor.   
6.  Too many widgets: Given how easy it is to add various blog widgets to your blog, it’s tempting to go overboard.  Resist the temptation.  In the early weeks, focus on the basics:  Content, comments, categories and perhaps a blog roll.  Save the fancy stuff for later.  
7.  Monetizing too early: Just because Google makes it easy to throw ads up on your blog doesn’t mean you should.  With minimal traffic, the amount of money you are going to make is negligible (probably pennies a day).  If you’re serious about blogging, you’ll first focus on building an audience before even attempting to try to make money at it.  Trying ads out on your blog too early will reduce the chances of your getting that critical early traffic.
 
You don’t have to blunder in the beginning of your blogging efforts.  HubSpot’s training and support program make it super simple and easy for you.”

This is so true even today, now that Google has changed the way to searched and gives relevance to blogs.  Changes in the way to bring blogs to market are in order if you are starting today.  If you already have a well established blog that is getting good traffic, then I would not touch it and start a second blog and grow that for the new ways fo the blogging world.

Foursquare vs. Gowalla location based mobile side by side look.

I was talking to a CMO friend about the difference between Foursquare and Gowalla.  Now she had not heard of either of these, thus a chart is worth a thousand words and Mashable did a wonderful job with a compare and contrast for the blog below. Note that is a few months old and the area has grown up a lot in the past few months, so it is a must be in area for companies and individuals. 

 

You can compete for mayorships of your favorite game or topic. Being the mayor has become the new hey are you on Twitter, that was the buzz last year. Also look at Brightkite, Loopt and Yelp in this space.

 

Foursquare vs. Gowalla: Location-Based Throwdown

foursquare vs gowalla imageBased in New York City, Shane Snow is a graduate student in Digital Media at Columbia University and founder of Scordit.com. He’s fascinated with all things geeky, particularly social media and shiny gadgets he’ll never afford.

 Just when you thought you had enough social networks in your life, two hot new ones are vying for your attention. Combine the benefits of sites like Yelp(), Twitter(), and Google Latitude, add in social gaming and some privacy measures, and you have the recipe that Foursquare(), the app that’s been called “next year’s Twitter,” by Mashable’s() own Pete Cashmore, and its chief competitor, Gowalla(), are cooking. Each has attracted a rapidly growing user base in 2009, and each is rushing madly to beat the other to your smartphone in 2010.


A Quick History


Many of the world’s great inventions – television for example – have been developed in parallel, unbeknown to their authors. Both Foursquare and Gowalla started development in late 2008, and both launched in Austin at SXSW in March this year. At the time, Foursquare stole the show, drowning out any fanfare that the then-buggy Gowalla tried to muster.

When asked if he was nervous when he heard about Foursquare’s simultaneous launch, Gowalla founder Josh Williams said, “On the contrary, I felt like the presence of other innovative location services was simply confirmation that we might be onto something interesting.”

Nine months and millions of dollars of funding later, Gowalla has positioned itself as a serious contender in the space. Bugs have been ironed out, and according to Williams, Gowalla is now seeing 20,000 checkins a day. While each app has its own specialties, they share some remarkable similarities, even in the liner notes:

foursquare vs gowalla about image

 I’m not sure who ripped off whom on this one, but at least we all love everyone!


Which Should You Use?


Regardless of who developed what first, these networks may soon work their way into your own social circles. The big question: Which should you use?

 The answer may simply be, “Whichever one your friends are using.” But if you’re the early adopter in the group, the blow-by-blow comparison below can help you sway your social circle to sign up for the one that fits you best.

foursquare vs gowalla chart

Cells shaded in orange indicate a feature that favors Gowalla, those in blue favor Foursquare, while crosshatching indicates a toss-up. However, opinions will vary on this, so please tell us what you think in the comments.

 As you can see, Gowalla trumps Foursquare in design and availability, but Foursquare takes the cake when it comes to check-in location accuracy, device support, friend management, and value added features like a city guide, and to-do list. Both apps are equally impressive in the way they dole out in-game and real-life perks and rewards.


The Problem with Gowalla


While Gowalla is very cool looking, my main beef is its tendency to emphasize you more, while Foursquare emphasizes your friends. One of the best things these apps have over other social networks is the ability to meet and interact with new, real people, meet up physically with your actual friends, and get recommendations from people you trust. I’m not saying you can’t do this with Gowalla, just that Foursquare seems to do it better. However, for those who enjoy the Pokemon-style gaming aspect, Gowalla with its pins and item swapping, is a definite first choice.

 Also, likely because of Gowalla’s slow start, although the service is available everywhere, even in New York City, where I live I rarely check in somewhere where an actual Gowalla user is also checked in. I bump into Foursquare users all over the place, though. If this were the board game Risk, Foursquare would be the guy who’s got 1,000 armies in Australia(Australia) while Gowalla has one army everywhere else. Unless Gowalla is really lucky with the dice, it’s probably only a matter of time before it gets edged out everywhere else too.

 As the location-based battle rages on, keep an eye on semi-competitors like Loopt and Brightkite(Brightkite) who may soon enter this particular fray. “I think both Foursquare and Gowalla are great apps – using the real world as the backdrop for a game is a lot of fun,” Loopt CEO Sam Altman recently confessed. “It’s probably a safe assumption that we’ll add some gaming elements.”

See also: 6 Tips for Getting the Most Out of Foursquare

Creating a sitemap on Google, Bing, Yahoo and Ask.

XML Sitemap—or sitemap is a list of the pages on your website, each search engine needs it to their specifications and requirements.  When you create and submit a sitemap helps make sure that Google, Yahoo, Bing and Ask know about all the pages on your site, including URLs that may not be  found by the  normal crawling process.  Having a site map for your website it a key to getting searched by the various search engine bots that I have written about in a previous blog about the 5 kinds of  Search Bots.  Your webmaster or agency should have all this information at their finger tips, if they do not,  you may not be getting the proper search engines you want going to your site.

Knowing what to ask, beyond… do we have a sitemap or do we have an XML feed needs this knowledge.  Ask if they have done all the formatting needed to have the XML sitemap work on all the search engines is key.  Having all the information at your finger tips to get the search engines to the search engines and what they need is priceless. 

Here is the data that I use from each one of the search engines as to what they require and how to:

____________________________________________________________________

Creating and submitting sitemaps to Google

The Google site says:

“About Sitemaps

Sitemaps are a way to tell Google about pages on your site we might not otherwise discover. In its simplest terms, a XML Sitemap—usually called Sitemap, with a capital S—is a list of the pages on your website. Creating and submitting a Sitemap helps make sure that Google knows about all the pages on your site, including URLs that may not be discoverable by Google’s normal crawling process.”

In addition to regular Sitemaps, you can also create Sitemaps designed to give Google information about specialized web content, including video, mobile, News, Code Search, and geographical (KML) information.

When you update your site by adding or removing pages, tell Google about it by resubmitting your Sitemap. Google doesn’t recommend creating a new Sitemap for every change.”
______________________________________________________________________________

Bing Webmaster Tools

To submit an XML-based Sitemap to Bing:

Step 1: Copy and paste the entire URL below as a single URL into the address bar of your browser:

     http://www.bing.com/webmaster/ping.aspx?sitemap=www.YourWebAddress.com/sitemap.xml

Step 2: Change “www.YourWebAddress.com” to your domain name

Step 3: Press ENTER”

_________________________________________________________________________

How Yahoo! supports sitemaps

“How Yahoo! Supports Sitemaps

Yahoo! supports the Sitemaps format and protocol as documented on www.sitemaps.org .

You can provide us a feed in the following supported formats. We recognize files with a .gz extension as compressed files and decompress them before parsing.

  • RSS 0.9, RSS 1.0, or RSS 2.0, for example, CNN Top Stories
  • Sitemaps, as documented on  www.sitemaps.org
  • Atom 0.3, Atom 1.0, for example, Yahoo! Search Blog
  • A text file containing a list of URLs, each URL at the start of a new line. The filename of the URL list file must be urllist.txt. For a compressed file the name must be urllist.txt.gz.
  • Yahoo! supports feeds for mobile sites. Submitting mobile Sitemaps directs our mobile search crawlers to discover and crawl new content for our mobile index. When submitting a feed that points to content on the mobile web, please indicate whether the encoding of the content was done with xHTML or WML.

    In addition to submitting your Sitemaps to Yahoo! Search through Yahoo! Site Explorer, you can:

  • Send an HTTP request. To send an HTTP request, please send using our ping API.
  • Specify the Sitemap location in your site’s robots.txt file. This directive is independent of the user-agent line, so you can place it wherever you like in your file.
  • Yahoo! Search will retrieve your Sitemap and make the URLs available to our web crawler. Sitemaps discovered by these methods cannot be managed in Site Explorer and will not show in the list of feeds under a site.

    Note: Using Sitemap protocol supplements the other methods that we use to discover URLs. Submitting a Sitemap helps Yahoo! crawlers do a better job of crawling your site. It does not guarantee that your web pages will be included in the Yahoo! Search index.”

    ___________________________________________________________________

    Ask.com webmaster FAQ

    “Web Search

    The Ask.com search technology uses semantic and extraction capabilities to recognize the best answer from within a sea of relevant pages. Instead of 10 blue links, Ask delivers the best answer to user’s questions right at the top of the page. By using an established technique pioneered at Ask, our search technology uses click-through behavior to determine a site’s relevance and extract the answer. Unlike presenting text snippets of the destination site, this technology presents the actual answer to a user’s question without requiring an additional click through. Underpinning these advancements are Ask.com’s innovative DADS, DAFS, and AnswerFarm technologies, which break new ground in the areas of semantic search, web extraction and ranking. These technologies index questions and answers from numerous and diversified sources across the web. It then applied its semantic search technology advancements in clustering, rephrasing, and answer relevance to filter out insignificant and less meaningful answer formats. In order to extract and rank exciting answers, as opposed to merely ranking web pages, Ask.com continues to develop a unique algorithms and technologies that are based on new signals for evaluating relevancy specifically tuned to questions.

    The Ask Website Crawler FAQ

    Ask’s Website crawler is our Web-indexing robot (or crawler/spider). The crawler collects documents from the Web to build the ever-expanding index for our advanced search functionality at Ask and other Web sites that license the proprietary Ask search technology.

    Ask search technology is unique from any other search technology because it analyzes the Web as it actually exists — in subject-specific communities. This process begins by creating a comprehensive and high-quality index. Web crawling is an essential tool for this approach, and it ensures that we have the most up-to-date search results.

    On this page you’ll find answers to the most commonly asked questions about how the Ask Website crawler works. For these and other Webmaster FAQs, visit our Searchable FAQ Database.

    Frequently Asked Questions

    1. What is a website crawler?
    2. Why does Ask use a website crawler?
    3. How does the Ask crawler work?
    4. How frequently will the Ask Crawler index pages from my site?
    5. Can I prevent Teoma/Ask search engine from showing a cached copy of my page?
    6. Does Ask observe the Robot Exclusion Standard?
    7. Can I prevent the Ask crawler from indexing all or part of my site/URL?
    8. Where do I put my robots.txt file?
    9. How can I tell if the Ask crawler has visited my site/URL?
    10. How can I prevent the Ask crawler from indexing my page or following links from a particular page?
    11. Why is the Ask crawler downloading the same page on my site multiple times?
    12. Why is the Ask crawler trying to download incorrect links from my server? Or from a server that doesn’t exist?
    13. How did the Ask Website crawler find my URL?
    14. What types of links does the Ask crawler follow?
    15. Can I control the rate at which the Ask crawler visits my site?
    16. Why has the Ask crawler not visited my URL?
    17. Does Ask crawler support HTTP compression?
    18. How do I register my site/URL with Ask so that it will be indexed?
    19. Why aren’t the pages the Ask crawler indexed showing up in the search results?
    20. Can I control the crawler request rate from Ask spider to my site?
    21. How do I authenticate the Ask Crawler?
    22. Does Ask.com support sitemaps?
    23. How can I add Ask.com search to my site?
    24. How can I get additional information?

     

    Q: What is a website crawler?
    A: A website crawler is a software program designed to follow hyperlinks throughout a Web site, retrieving and indexing pages to document the site for searching purposes. The crawlers are innocuous and cause no harm to an owner’s site or servers.

    Q: Why does Ask use website crawlers?
    A: Ask utilizes website crawlers to collect raw data and gather information that is used in building our ever-expanding search index. Crawling ensures that the information in our results is as up-to-date and relevant as it can possibly be. Our crawlers are well designed and professionally operated, providing an invaluable service that is in accordance with search industry standards.

    Q: How does the Ask crawler work?

    • The crawler goes to a Web address (URL) and downloads the HTML page.
    • The crawler follows hyperlinks from the page, which are URLs on the same site or on different sites.
    • The crawler adds new URLs to its list of URLs to be crawled. It continually repeats this function, discovering new URLs, following links, and downloading them.
    • The crawler excludes some URLs if it has downloaded a sufficient number from the Web site or if it appears that the URL might be a duplicate of another URL already downloaded.
    • The files of crawled URLs are then built into a search catalog. These URL’s are displayed as part of search results on the site powered by Ask’s search technology when a relevant match is made.

     

    Q: How frequently will the Ask Crawler download pages from my site?
    A: The crawler will download only one page at a time from your site (specifically, from your IP address). After it receives a page, it will pause a certain amount of time before downloading the next page. This delay time may range from 0.1 second to hours. The quicker your site responds to the crawler when it asks for pages, the shorter the delay.

    Q. Can I prevent Teoma/Ask search engine from showing a cached copy of my page?
    A: Yes. We obey the “noarchive” meta tag. If you place the following command in your HTML page, we will not provide an archived copy of the document to the user.

    < META NAME = “ROBOTS” CONTENT = “NOARCHIVE” >

    If you would like to specify this restriction just for Teoma/Ask, you may use “TEOMA” in place of “ROBOTS”.

    Q: Does Ask observe the Robot Exclusion Standard?
    A: Yes, we obey the 1994 Robots Exclusion Standard (RES), which is part of the Robot Exclusion Protocol. The Robots Exclusion Protocol is a method that allows Web site administrators to indicate to robots which parts of their site should not be visited by the robot. For more information on the RES, and the Robot Exclusion Protocol, please visit http://www.robotstxt.org/wc/exclusion.html.

    Q: Can I prevent the Ask crawler from indexing all or part of my site/URL?
    A: Yes. The Ask crawler will respect and obey commands that direct it not to index all or part of a given URL. To specify that the Ask crawler visit only pages whose paths begin with /public, include the following lines:

    # Allow only specific directories
    User-agent: Teoma
    Disallow: /
    Allow: /public

     

    Q: Where do I put my robots.txt file?
    A: Your file must be at the top level of your Web site, for example, if http://www.mysite.com is the name of your Web site, then the robots.txt file must be at http://www.mysite.com/robots.txt.

    Q: How can I tell if the Ask crawler has visited my site/URL?
    A: To determine whether the Ask crawler has visited your site, check your server logs. Specifically, you should be looking for the following user-agent string:

    User-Agent: Mozilla/2.0 (compatible; Ask Jeeves/Teoma)

     

    Q: How can I prevent the Ask crawler from indexing my page or following links from a particular page?
    A: If you place the following command in the section of your HTML page, the Ask crawler will not index the document and, thus, it will not be placed in our search results:

    < META NAME = “ROBOTS” CONTENT = “NOINDEX” >

    The following commands tell the Ask crawler to index the document, but not follow hyperlinks from it:

    < META NAME = “ROBOTS” CONTENT = “NOFOLLOW” >

    You may set all directives OFF by using the following:

    < META NAME = “ROBOTS” CONTENT = “NONE” >

    See http://www.robotstxt.org/wc/exclusion.html#meta for more information.

    Q: Why is the Ask crawler downloading the same page on my site multiple times?
    A: Generally, the Ask crawler should only download one copy of each file from your site during a given crawl. There are two exceptions:

    • A URL may contain commands that “redirect” the crawler to a different URL. This may be done with the HTML command:
      < META HTTP-EQUIV=”REFRESH” CONTENT=”0; URL=http://www.your page address here.html” >

      or with the HTTP status codes 301 or 302. In this case the crawler downloads the second page in place of the first one. If many URLs redirect to the same page, then this second page may be downloaded many times before the crawler realizes that all these pages are duplicates.

    • An HTML page may be a “frameset.” Such a page is formed from several component pages, called “frames.” If many frameset pages contain the same frame page as components, then the component page may be downloaded many times before the crawler realizes that all these components are the same.

     

    Q: Why is the Ask crawler trying to download incorrect links from my server? Or from a server that doesn’t exist?
    A: It is a property of the Web that many links will be broken or outdated at any given time. Whenever any Web page contains a broken or outdated link to your site, or to a site that never existed or no longer exists, Ask will visit that link trying to find the Web page it references. This may cause the crawler to ask for URLs which no longer exist or which never existed, or to try to make HTTP requests on IP addresses which no longer have a Web server or never had one. The crawler is not randomly generating addresses; it is following links. This is why you may also notice activity on a machine that is not a Web server.

    Q: How did the Ask Website crawler find my URL?
    A: The Ask crawler finds pages by following links (HREF tags in HTML) from other pages. When the crawler finds a page that contains frames (i.e., it is a frameset), the crawler downloads the component frames and includes their content as part of the original page. The Ask crawler will not index the component frames as URLs themselves unless they are linked via HREF from other pages.

    Q: What types of links does the Ask crawler follow?
    A: The Ask crawler will follow HREF links, SRC links and re-directs.

    Q. Can I control the rate at which the Ask crawler visits my site?
    A. Yes. We support the “Crawl-Delay” robots.txt directive. Using this directive you may specify the minimum delay between two successive requests from our spider to your site.

    Q: Why has the Ask crawler not visited my URL?
    A: If the Ask crawler has not visited your URL, it is because we did not discover any link to that URL from other pages (URLs) we visited.

    Q: Does Ask crawler support HTTP compression?
    A: Yes, it does. Both HTTP client and server should support this for the HTTP compression feature to work. When supported, it lets webservers send compressed documents (compressed using gzip or other formats) instead of the actual documents. This would result in significant bandwidth savings for both the server and the client. There is a little CPU overhead at both server and client for encoding/decoding, but it is worth it. Using a popular compression method such as gzip, one could easily reduce file size by about 75%.

    Q: How do I register my site/URL with Ask so that it will be indexed?
    A: We appreciate your interest in having your site listed on Ask.com and the Ask.com search engine. Your best bet is to follow the open-format Sitemaps protocol, which Ask.com supports. Once you have prepared a sitemap for your site, add the sitemap auto-discovery directive to robots.txt, or submit the sitemap file directly to us via the ping URL. (For more information on this process, see Does Ask.com support sitemaps?) Please note that sitemap submissions do not guarantee the indexing of URLs.

    Create your Web site and set up your Web server to optimize how search engines look at your site’s content, and how they index and trigger based upon different types of search keywords. You’ll find a variety of resources online that provide tips and helpful information on how to best do this.

    Q: Why aren’t the pages the Ask crawler indexed showing up in the search results at Ask.com?
    A: If you don’t see your pages indexed in our search results, don’t be alarmed. Because we are so thorough about the quality of our index, it takes some time for us to analyze the results of a crawl and then process the results for inclusion into the database. Ask does not necessarily include every site it has crawled in its index.

    Q: Can I control the crawler request rate from Ask spider to my site?
    A: Yes. We support the “Crawl-Delay” robots.txt directive. Using this directive you may specify the minimum delay between two successive requests from our spider to your site.

    Q. How do I authenticate the Ask Crawler?
    A: A. User-Agent is no guarantee of authenticity as it is trivial for a malicious user to mimic the properties of the Ask Crawler. In order to properly authenticate the Ask Crawler, a round trip DNS lookup is required. This involves first taking the IP address of the Ask Crawler and performing a reverse DNS lookup ensuring that the IP address belongs to the ask.com domain. Then perform a forward DNS lookup with the host name ensuring that the resulting IP address matches the original.

    Q: Does Ask.com support sitemaps?
    A: Yes, Ask.com supports the open-format Sitemaps protocol. Once you have prepared the sitemap, add the sitemap auto-discovery directive to robots.txt as follows:

    SITEMAP: http://www.the URL of your sitemap here.xml

    The sitemap location should be the full sitemap URL. Alternatively, you can also submit your sitemap through the ping URL:

    http://submissions.ask.com/ping?sitemap=http%3A//www.the URL of your sitemap here.xml

    Please note that sitemap submissions do not guarantee the indexing of URLs. To learn more about the protocol, please visit the Sitemaps web site at http://www.sitemaps.org.

    Q: How can I add Ask.com search to my site?
    A: We’ve made this easy, you can generate the necessary code here.

    Q: How can I get additional information?
    A: Please visit our full Searchable FAQ Database.”