Google has strict policies against what's known as
cloaking: showing one web page to the crawler that indexes it but then a different page to a user. We do this so that users aren't deceived into clicking through to a site that's not what they were expecting. While the anti-cloaking policies are important for users, they do create some challenges for publishers who charge for content. Our crawlers can't fill out a registration or payment form to see what's behind a site's paywall, but they need access to the information in order to index it.
One way we overcome this is through a program called First Click Free. Participating publishers allow the crawler to index their subscription content, then allow users who find one of those articles through Google News or Google Search to see the full page without requiring them to register or subscribe. The user's first click to the content is free, but when a user clicks on additional links on the site, the publisher can show a payment or registration request. First Click Free is a great way for publishers to promote their content and for users to check out a news source before deciding whether to pay. Previously, each click from a user would be treated as free. Now, we've
updated the program so that publishers can limit users to no more than five pages per day without registering or subscribing. If you're a Google user, this means that you may start to see a registration page after you've clicked through to more than five articles on the website of a publisher using First Click Free in a day. We think this approach still protects the typical user from cloaking, while allowing publishers to focus on potential subscribers who are accessing a lot of their content on a regular basis.
In addition to First Click Free, we offer another solution: We will crawl, index and treat as "free" any preview pages - generally the headline and first few paragraphs of a story - that they make available to us. This means that our crawlers see the exact same content that will be shown for free to a user. Because the preview page is identical for both users and the crawlers, it's not cloaking. We will then label such stories as "subscription" in Google News. The ranking of these articles will be subject to the same criteria as all sites in Google, whether paid or free. Paid content may not do as well as free options, but that is not a decision we make based on whether or not it's free. It's simply based on the popularity of the content with users and other sites that link to it.
These are two of the ways we allow publishers to make their subscription content discoverable, and we're going to keep talking with publishers to refine these methods. After all, whether you're offering your content for free or selling it, it's crucial that people find it. Google can help with that.
Posted by Josh Cohen, Senior Business Product Manager
I define my usage of the internet into two epochs: before and after Google, so I am a fan.
ReplyDeleteBUT: when I search Google I expect to find content that is freely and fully downloadable as is the norm on the web.
Increasingly I have found the top links on some searches are unsatisfactory because they require subscription or registration, neither of which I desire.
If I do subscribe to online content, it will be because I decide a resource is valuable to me on a long term basis, and not because I lighted there once after a Google search.
I do NOT want pay-for content in my top search results on a routine Google search. I want, for philosophical and political reasons, to find only content that is world-readable, because that is the only content I want to cite. To put it another way, I want to be as ignorant as everyone else.
If too much pay-for content arrives in the top-ranked links following a Google search (and this seems to happening increasingly) I will use one of the many other search engines available.
I think you know this, and it keeps you straight, but I am concerned fiduciary pressures will corrupt you one day, and I assure you I follow the issue closely.