Direct Navigation and error correction are becoming significant areas of web development, enhancing the Internet surfing experience for users, enabling ISPs to monetise hitherto error traffic, helping brands to grow traffic through keywords and building advertising revenues for PPC and CPA players through the delivery of qualified direct navigation traffic. Today the PPC sponsored click through or the pay for transaction approach is the fiscal model behind all of the current error and keyword redirection companies such as Golog, Barefruit and Paxfire.
Originally pioneered by RealNames in 1998, the user entered a keyword and was taken directly via a paid-for route to a specific web page. Hence ‘jobs’ would be bought by Jobs.com for $1m. Each time the word ‘jobs’ was typed in the browser address bar of IE4, the user was taken to jobs.com website via an interstitial advising the user that they were navigating by a RealNames keyword. RealNames had a database of over 1.5m keywords, all of which were paid for. They were both brand names and a limited number of paid-for generic terms. The user of generics was restricted by MSN to use by them to navigate to MSN results pages. Other examples of direct navigation technology existed including the AOL keywords and Common Names.
The raison d’être of direct navigation was that a study by MIT in 2000 revealed that each time a user was forced to click to the next page to uncover a particular location, 50% of the audience was lost. Hence if a deep site location was 5 clicks away for every 1000 users searching for the initial site only 31 users would follow through. A subsequent study by the University of Boston proved that direct navigation improved eCommerce conversion by between 65% and 300% depending on category .
RealNames was forced into Chapter 7 by MSN because they viewed the rapid growth of keyword use as damaging their search business. Between October 2001 and May 2002 RealNames PPVs rose from 2m per month to 30m per month. The growth was 75% per month.
The direct navigation approach used by RealNames was successful because it had near 100% of user coverage through the embedding with Microsoft IE3 and IE4. This was also its main Achilles heel; since MSN were able to terminate this exclusive contract.
Since 2002 there have been a number of direct navigation solutions based on keywords that have been deployed. Most notably AOL continues with AOL keywords (although their use is restricted AOL subscribers). Yahoo! has deployed their own version on Yahoo! enabled browsers, as have Firefox and Opera.
Two years ago Google moved to introduce a ‘paid-for’ keyword direct navigation system based on the RealNames approach with the so-called “Browse by Name” approach, as well as a browser based spelling correction service using stored URL data. Google offered both Brands and Generics with direct navigation. Since it used the Google enable browser they have dispensed with any interstitial page. Type “Ford Explorer” in Google Browser and it will navigate directly to the deep site Ford Explorer page. Similarly type “Mortgages” it will directly navigate to “UK Mortgages Online”. All these solutions are browser-based. Google moved away from monetising error traffic but maintained its direct navigation approach with it's Google Toolbar and through a commercial relationship with Mozilla to apply direct navigation to results and brand sites to browser bar search on Firefox. The only other major keyword driven system was deployed by J-Word in Japan, which is based on 3721 company's downloadable system. 3721, a Chinese company, was acquired by Yahoo! in 2003 for $120m.
Error correction as a result of mis-spelt domain names and broken links in search results, resulting from 400 and 500 class errors are relatively new phenomena. Initial approaches that have been adopted were limited to the analysis of non existent domains (NXDomain). The first to be introduced was SiteFinder from Verisign using “wild card” technology provided by Paxfire in the US. This technology identified when a non-existent domain was being sought and offered the user the opportunity to buy the erroneous input. However, the service responded to both the “A” record domain name request to the authoritative name servers and “Mx” record which is associated with email. Providing a positive answer to a incorrect email address resulted in opening ISPs SMTP mail systems to virus attack. The service was quickly abandoned after a formal request by ICANN , the Internet’s governing body. This experience has made ISPs very suspicious of any DNS “modifications”. However, the level of NXDomain traffic has been identified as being a significant source of error traffic. Total network error traffic is estimated by Verisign to account for approximately 15% of total network traffic of which NXDomain traffic makes up about 20%. A recent study by Google recently indicated that 404 HTTP errors alone account for 6.96% of published pages on average of the 11.5 billion web pages currently published on the web or some 800 million error pages!
A more sophisticated approach is being rolled out by a British company called Barefruit. Barefruit is aiming to provide a relevant result for nearly 100% of network error traffic by not only dealing with the mis-spelt URL error resulting in an NXDomain response, but also covering the 400 and 500 class errors.
The Barefruit Service
Barefruit has a service which is based on the evaluation of broken links in search results and the identification of DNS errors. It can resolve 400 and 500 errors in particular the eponymous 404 Error as well as isolate NXDomain errors. It does this by utilising a proxy server to store a temporary cache of the page prior to the display of the error page. It then detects the error response and then analyses the broken link data and provides a further set of links rather than an error page. Barefruit estimate that a typical broadband user will generate fifty errors every month.
Barefruit patented solution covers the broadest range of error traffic.
1 Mis-spelled urls (NXDomain)
2 Keywords
3 400 Errors
4 500 Errors
The addition, of direct navigation methodology is relatively simple to achieve by combining the NXDomain redirection patches currently in the public domain with an extended taxonomy.
It differs from the previous direct navigation approaches in that it works at the ISP level and not the browser level, although a browser download is available, only deploying when an NXDomain event occurs in the browser or when a 400 or 500 error occurs in a link.
The Barefruit solution provides the ISP with an opportunity to take back a degree of control over “their” user traffic rather than losing it to third parties and also creates a revenue stream from users clicking through on sponsored links. One significant fillup for Barefruit is the rapid adoption of DPI - Deep Packet Inspection, which allows broadband and cable operators to monitor their web traffic and so identify http errors relatively easily and allow Barefruit to analyse and identify user preferences and hence present highly relevant results instead of errors.
Other Developments
Other developments in ‘Direct Navigation and Error correction’ are being pursued by a number of players although none currently offer a solution as comprehensive as Barefruit.
Google has launched its own keyword system in ‘Beta’ which is a direct click through system running from the address bar of Google enabled browsers. They are also introducing spelling correction at the browser bar level for mis-spelled URLs of the URL data stored in their database. This operates in a predictive mode similar to the methodology used in mobile phone text services. It does not eliminate mis-spellings but does reduce MURL events. The approach is not DNS based but its deployment in Europe is limited by the need to comply with EU Directive EU 2002/53. It also tends to offer the ISP with limited income opportunities.
Yahoo! is also developing a direct navigation service similar to Google’s ‘I feel lucky’ approach, taking users directly to the sponsored website. This is likely to be deployed through Yahoo! enabled browsers and their ISP partners such as BT Yahoo! in the UK. BT Yahoo! enable browsers already use keyword navigation utilising direct navigation. Yahoo! will use a variation in their T&Cs to overcome the issues associated with the EU Directive.
Both Yahoo! and Google suffer from lack of independence in respect of ISPs given Overture’s ownership by Yahoo! and relationship with BT Yahoo! and Google’s recent link up with AOL.
Paxfire, a US based company, has been promoting its mis-spelled URL service to ISPs. The service is based on a proxy server solution and relies on a data update solution similar to the ‘black listing’ of sites deployed by website filtering solutions. It was this service which was the basis of the Verisign SiteFinder service and has been demonstrated to be difficult to configure successfully to avoid key issues. Paxfire's Look-Up Service allows ISPs to capture search terms and misspelled URLs entered into any Internet web browser address bar and deliver relevant content and advertising. Paxfire's technology works at the network level and provides a plug and play platform that requires installation of the Paxfire Look-Up Engine (PLE) in front of a DNS server in the ISP’s network.
Golog is limited to NXDomain events and the current business model in Europe and appears to rely on a “forced redirection” after 4 to 7 seconds. Golog installs a software patch into an ISPs DNS infrastructure which redirects the NXDomain to a Golog analysis server which identifies a key term which is then looked-up against a Golog database and where appropriate Golog monetises its mis-spelled URLs through partnership with Miva (formerly eSpotting)or a PPA partner such as Zanox. The Golog solution does not cache the request and cannot deal with any 400 or 500 error traffic. There are some risks associated with the “forced rediection” approach as it can result in the user feeling that they have received an unrelated result which is a poor user experience and can result in the advertiser being charged for a click that a user hasn’t selected.
Conclusion
The development and interest in direct navigation and error correction is growing at a significant rate. In the competitive on-line advertising market, its user ability and untapped potential, it has the potential to supersede the current listings and search approach as users become more familiar with the ability to use the address bar as a direct navigation tool combined with auto error correction both at the browser and in the search listings. One key aspect of any solution that an ISP may apply lies in offering its customers a route to "opt-out" of any error redirection.
To date only Barefruit has developed a combination of direct navigation and error correction which covers nearly 100% of error traffic and it also enables the ISPs to be offered a very powerful revenue generating service from what had previously been regarded as rubbish traffic.
Showing posts with label DNS. Show all posts
Showing posts with label DNS. Show all posts
Thursday, September 20, 2007
Sunday, April 23, 2006
Interactive, Networked and err….404 File not found!
It was reported recently that Britain had overtaken France as the largest user of broadband lines in Europe, according to the internet consultancy Point Topic. Today it is estimated that the UK has 9.8m broadband lines, France has 9.7m and Germany is in third place. During the same period we have seen massive growth in the online advertising market with the UK accounted for the largest share of European online ad-spend, at 39% of the total (€3.101 billion).
Typically broadband is 40 times faster than ordinary dial-up and the speed of broadband tends to be addictive, so it is doubly frustrating when you come to a screeching halt due to a browsing error. If you're running any of the popular internet browsers such as Microsoft’s Internet Explorer or Mozilla’s Firefox, you are most likely to get "The page cannot be displayed" followed by largely worthless advice that's incomprehensible to the average surfer, which concludes with "Cannot find server or DNS Error." or reference to some numbered code such “error 404” . Network Solutions Inc (NSI), the domain name registry for .com, .net and .org estimate that up to 15% of all network traffic is this sort of error traffic. Also a recent study by Google recently indicated that 404 HTTP errors alone account for 6.96% of published pages on average of the 11.5 billion web pages currently published on the web or some 800 million error pages! These huge volumes of errors and the demand for more advertising inventory has stimulated significant interest in monetising this massive untapped market characterised by one senior Ad Executive as “the last white space”.
In the past the main beneficiary of much of this traffic has been Microsoft and their MSN Search Engine. The error codes are identified by the Internet Explorer browser and the user is delivered to an MSN Search result. Consequently this error traffic has managed to boost MSN Search into second place behind Google in the Global Search Engine stakes.
Search Engines have long been aware of the value of this traffic and it will come as little surprise that the behemoth Google has had a product called “AdSense for Errors”. Google extracted contextually related keywords and advertisements from the DNS error pages by using Google's semantic technology to "understand" the meaning of each error. It tended to focus on a class of error known as an NXD or Non eXistent Domain, the error generated when a mis-spelt or unregistered domain name is typed into the browser address bar. This has evolved into Adsense for Domains focusing on the domain name business. However, Google hasn’t left the error space entirely to Microsoft, in recent years it has rolled out the Google Toolbar to “hi-jack” error traffic such as search from the browser address bar directly to Google search results in the guise of did you mean. In case you thought couldn’t possibly be any money in errors, Google has reportedly paid Mozilla millions of dollars to ensure that address bar search defaults to Google results rather than go to a DNS error page!
Verisign, the owner of NSI, got into a spat with ICANN (www.ICANN.org), the Internet’s Governing body, when it started responding to these NXD mis-spellings with a so-called “wild card response”, offering users the opportunity to buy the mis-spelt domain. Unfortunately, the service known as “SiteFinder” was poorly configured and resulted in opening many ISP and Enterprise email servers to spam attacks.
Now a British company called Barefruit (www.barefruit.co.uk) has emerged to become the Global leader in this developing error space. Barefruit resolves all the most commonly occurring errors, handling 98% of the error traffic including NXDomains, 400 Website errors including the aforementioned 404 File not found error and 500 Server errors. The company has successfully patented a technique of caching the user’s desired link and if an error occurs, analysing both the domain name information and any associated description and directing the user to either an exact match for a mis-typed brand or trade mark or providing a contextual search result to allow the user to make an informed choice. Through relationships with a broad range of databases and search partners, Barefruit delivers both organic and sponsored listings and claims to offer a near 100% relevancy of the user’s desired destination.
The Barefruit solution is available in a variety of formats which are simple to install either at the user level, via a plug-in, or in partnership with an Internet Service Provider (ISP) through modified DNS software or through the installation of a proxy server. The service also works with every browser type. The benefit of the Barefruit service is not only limited to the resolution of the error, but also provides the user the ability to navigate the Internet by the use of Keywords enabling every ISP with a “direct navigation” capability similar to AOL Keywords or Google’s “Browse by Name”.
With Barefruit, there is also a financial benefit to the ISP as they share the revenue generated by their users clicking on sponsored links. So what had been considered to be rubbish in the past and an annoyance in Internet navigation may now be turned to the benefit of users and ISPs alike. This has led to an increasing number of Tier 1 ISPs, in the UK, Europe and in the US market to adopt Barefruit’s solution. to enhance the Internet surfing experience of millions of users, monetise hitherto error traffic. This new source of traffic is also keenly wanted by advertisers since with Barefruit’s ability to extract the desired requirements of the user is helping brands to grow traffic and continue to build advertising revenues for online advertised through the delivery of qualified traffic.
2007 has seen massive growth in the development of the error correction market and it is set to become the most significant area of development of the so-called Web 2.0. A number of new players have tried to enter the market to monetise this traffic, but only Barefruit has such a broad range of patented error solutions. Barefruit’s growing reputation for extracting relevance and literally “bearing fruit” in millions of pounds of additional revenue to ISPs and online advertisers alike whilst improving the Internet users surfing experience.
Typically broadband is 40 times faster than ordinary dial-up and the speed of broadband tends to be addictive, so it is doubly frustrating when you come to a screeching halt due to a browsing error. If you're running any of the popular internet browsers such as Microsoft’s Internet Explorer or Mozilla’s Firefox, you are most likely to get "The page cannot be displayed" followed by largely worthless advice that's incomprehensible to the average surfer, which concludes with "Cannot find server or DNS Error." or reference to some numbered code such “error 404” . Network Solutions Inc (NSI), the domain name registry for .com, .net and .org estimate that up to 15% of all network traffic is this sort of error traffic. Also a recent study by Google recently indicated that 404 HTTP errors alone account for 6.96% of published pages on average of the 11.5 billion web pages currently published on the web or some 800 million error pages! These huge volumes of errors and the demand for more advertising inventory has stimulated significant interest in monetising this massive untapped market characterised by one senior Ad Executive as “the last white space”.
In the past the main beneficiary of much of this traffic has been Microsoft and their MSN Search Engine. The error codes are identified by the Internet Explorer browser and the user is delivered to an MSN Search result. Consequently this error traffic has managed to boost MSN Search into second place behind Google in the Global Search Engine stakes.
Search Engines have long been aware of the value of this traffic and it will come as little surprise that the behemoth Google has had a product called “AdSense for Errors”. Google extracted contextually related keywords and advertisements from the DNS error pages by using Google's semantic technology to "understand" the meaning of each error. It tended to focus on a class of error known as an NXD or Non eXistent Domain, the error generated when a mis-spelt or unregistered domain name is typed into the browser address bar. This has evolved into Adsense for Domains focusing on the domain name business. However, Google hasn’t left the error space entirely to Microsoft, in recent years it has rolled out the Google Toolbar to “hi-jack” error traffic such as search from the browser address bar directly to Google search results in the guise of did you mean. In case you thought couldn’t possibly be any money in errors, Google has reportedly paid Mozilla millions of dollars to ensure that address bar search defaults to Google results rather than go to a DNS error page!
Verisign, the owner of NSI, got into a spat with ICANN (www.ICANN.org), the Internet’s Governing body, when it started responding to these NXD mis-spellings with a so-called “wild card response”, offering users the opportunity to buy the mis-spelt domain. Unfortunately, the service known as “SiteFinder” was poorly configured and resulted in opening many ISP and Enterprise email servers to spam attacks.
Now a British company called Barefruit (www.barefruit.co.uk) has emerged to become the Global leader in this developing error space. Barefruit resolves all the most commonly occurring errors, handling 98% of the error traffic including NXDomains, 400 Website errors including the aforementioned 404 File not found error and 500 Server errors. The company has successfully patented a technique of caching the user’s desired link and if an error occurs, analysing both the domain name information and any associated description and directing the user to either an exact match for a mis-typed brand or trade mark or providing a contextual search result to allow the user to make an informed choice. Through relationships with a broad range of databases and search partners, Barefruit delivers both organic and sponsored listings and claims to offer a near 100% relevancy of the user’s desired destination.
The Barefruit solution is available in a variety of formats which are simple to install either at the user level, via a plug-in, or in partnership with an Internet Service Provider (ISP) through modified DNS software or through the installation of a proxy server. The service also works with every browser type. The benefit of the Barefruit service is not only limited to the resolution of the error, but also provides the user the ability to navigate the Internet by the use of Keywords enabling every ISP with a “direct navigation” capability similar to AOL Keywords or Google’s “Browse by Name”.
With Barefruit, there is also a financial benefit to the ISP as they share the revenue generated by their users clicking on sponsored links. So what had been considered to be rubbish in the past and an annoyance in Internet navigation may now be turned to the benefit of users and ISPs alike. This has led to an increasing number of Tier 1 ISPs, in the UK, Europe and in the US market to adopt Barefruit’s solution. to enhance the Internet surfing experience of millions of users, monetise hitherto error traffic. This new source of traffic is also keenly wanted by advertisers since with Barefruit’s ability to extract the desired requirements of the user is helping brands to grow traffic and continue to build advertising revenues for online advertised through the delivery of qualified traffic.
2007 has seen massive growth in the development of the error correction market and it is set to become the most significant area of development of the so-called Web 2.0. A number of new players have tried to enter the market to monetise this traffic, but only Barefruit has such a broad range of patented error solutions. Barefruit’s growing reputation for extracting relevance and literally “bearing fruit” in millions of pounds of additional revenue to ISPs and online advertisers alike whilst improving the Internet users surfing experience.
Subscribe to:
Posts (Atom)
