When will Next Generation Broadband start to deliver for the UK?

If we are to enjoy the plethora of digital services such as IPTV in a quality comparable to HD delivered via the internet we will need the bandwidth that only the Next Generation Broadband known as Fibre to the home (FTTH) can offer. FTTH projects are being rolled out around the world it seems the UK is lagging far behind our developed world competitors. Except for a small trail announced by BT Openreach in Kent which will target up to 9,500 new homes and offices in an area of 1,000 acres. This at a time in Japan where FTTH installations are poised to pass a now declining ADSL userbase. FTTH is a form of fibre-optic communication delivery in which an optical fibre is run directly onto the customers' premises. Traditionally only the network to the local exchange was fibre and methods such as copper wires or coaxial cable for "last mile" delivery.

The advantages of FTTH are huge.

For customers, it offers virtually unlimited bandwidth capacity and truly integrated voice, data, and video.

For providers, the ability to offer integrated services could mean higher profits and no electronics in the field means less maintenance. In addition, the costs of implementing technology are dropping and are similar to costs of deploying copper, but the payoff is much larger.

So how does FTTH work?

The Internet "backbone" is made up of fibre optic cables (very thin glass filaments) that have enormous bandwidth and use light pulses to carry information. Most customers, however, connect to the backbone through copper-based technologies like twisted pair and DSL or Hybrid Fibre Coax cable, which have limited bandwidth and limited capacity to carry integrated voice, video, and data services. This creates a speed and service bottleneck in the "last mile," the distance between the fibre optic backbone and customers.

Providers primarily offer FTTH through two types of architectures, point-to-point and passive optical network (PON). Point-to-point requires providers to install an optical transceiver in the provider's central office for each customer. PON uses a single transceiver with a splitter to serve up to 32 businesses and residential customers who share the bandwidth. The splitter is located up to 9km from the exchange, and a single strand of fibre can carry the signal another 1km to the customer. Once the fibre reaches the customer's home or business, an optical electrical converter (OEC) on the side of the building converts the optical signal to an electrical signal that can interface with existing copper wiring.

Some providers are also using Gigabit Ethernet over fibre to provide customers with broadband access.

Fibre optic cables can currently carry information at speeds greater than 2.5 gigabits per second. Residential/business FTTH typically offers speeds from 10 mbps to over 100 mbps, which is a hundred times faster than most cable or DSL service and over twice as fast as a T3 connection.

What is happening elsewhere?

In Europe, the majority of live fibre customers are in Italy, Sweden and Denmark. While a number of incumbents have announced plans fibre deployments, these have generally been for their backbone infrastructure rather than direct to the consumer - e.g. Deutsche Telekom's (DT) rollout of a €3.3bn fibre/VDSL network offers uses fibre for the back end and VDSL for the final connection to the home. DT expects the service to be live in 50 cities by year-end 2007. Many fibre rollouts in Europe are also still only in Greenfield and pilot stages like BT. Moreover, the majority of projects involving fibre are developed by utility and municipal organisations rather than the incumbents themselves.

Comparing Europe with the Japan shows how pathetic our efforts in the UK really are. By the middle of 2008, Japan’s MultiMedia Research Institute forecasts that FTTH installations are forecasted to surpass traditional ADSL reaching 19 million lines by 2009 out of a total instaled broadband base of 35 million. This suggests that by 2009, that the majority of the Japanese population will be accessing the new range of high definition IPTV, real-time video phone and a huge range of fast integrated services making the UK look like we are still using Prestel for our on-line service!

So What?

Ten years ago I wrote my Masters’ dissertation on the need to create a national fibre infrastructure taking fibre to the doorstep of every home and business in the UK. At the time I estimated the cost to be in the region of £24bn and that given the local loop or last mile to be considered a natural monopoly we should seek to establish a common business, such as BT Openreach is today, to set-up and run the infrastructure allowing for commercial organisations to deliver the range of new and exciting services. This network would form the basis of a new “Information Society”. Looking back at my cost estimate, this now seems to pale into insignificance given the money that government seems prepared to waste on supporting inefficient banks such as Northern Rock.

We never seem to have the will to take the risk of investing in infrastructure that are key to our economic wellbeing until we get to the point that we face the collapse of key services. Just look at our railways, The London Underground, even our airports and compare them to what you find in Europe and the Far East. The UK seems content to operate on a piecemeal second rate basis.

If the UK is to prosper in the 21^st Century, we will need to start to work in a strategic manner and not rely on the occasional prestige project such as the Channel Tunnel Rail Link to demonstrate our commitment to the future delivered ten years late and for twice the original projected cost.

UK ISP Numbers

As with the US ISP market previously, I have just finished an analysis of the DSL and Cable Access market in the UK. I worked with the ITU published data for Broadband usage numbers and Neilson Ratings to get an accurate picture as well all the reports and disclosures for each of the ISPs shown below. I believe these figures represent a reasonably accurate representation of the genuine adoption of broadband either via DSL or Cable. Broadband connections included in this data cover download speeds equal to or faster than 256kbit/s. Dial-up is collapsing very fast and very few of the ISPs offer any user numbers.

ISP		Total	% of UK Accts
BT		4,074,000	27.56%
Virgin Media		3,590,000	24.28%
CPW (inc AOL)		2,500,000	16.91%
Tiscali UK		2,000,000	13.53%
Orange		1,142,000	7.72%
Sky		1,000,000	6.76%
Kingston		140,851	0.95%
Thus		132,000	0.89%
Entanet		80,000	0.54%
Clara.net UK		72,000	0.49%
Breathe		20,000	0.14%
Supanet		8,000	0.05%
Others		26,000	0.18%
Total		14,784,851	100%

There has been considerable consolidation of the UK market over the past year. Most recently we have seen the acquisition by Tiscali of Homechoice and Pipex's user base, Sky's acquisition of Easynet and CPW acquiring the AOL userbase. With the entry last month of O2 into the fixed Broadband market with a very competitive offering and Vodafone flexing its muscles acquiring Tele 2's operations in Spain and Italy, the race to mop up the remaining minnows is likely to continue into 2008. In particular the continued independence of Thus and Kingston in this market must be considered to be in question. Maybe C&W with their improved fiscal results would see the Telecoms operations of Thus and Kingston a good fit, allowing for the broadband base to be sold to the aggressive Sky! Certainly 2008 is likely to be as exciting as the last 18 months and we may see the top six consolidate further with Sky possibly making a play for Tiscali and achieve their stated aim of 3 million users by the end of 2008.

US ISP and Cable Subscriber Numbers

I have recently been evaluating the US ISP market and trying to get accurate figures for Internet access for DSL, Cable Access and Dial-up. I worked with OECD published data for Broadband adoption and I have gone through all the reports and disclosures for each of the ISPs shown below. I believe these figures represent a reasonably accurate representation of the genuine adoption of broadband either via DSL or Cable. Broadband connections included in this data cover download speeds equal to or faster than 256kbit/s.

Table 1 US ISP & Cable Operators
Name	ADSL	Cable	Dial	Total
AOL			5,100,000	5,100,000
United Online	2,300,000			2,300,000
Road Runner		6,900,000		6,900,000
Earthlink	575,000		1,725,000	2,300,000
Embarq	1,100,000			1,100,000
Cox		3,500,000		3,500,000
Comcast		11,500,000		11,500,000
Charter	2,400,000			2,400,000
Cablevision		2,000,000		2,000,000
Suddenlink Comm		1,400,000		1,400,000
Bright House Networks		2,200,000		2,200,000
AT&T World net/BellSouth/SBC	10,500,000	6,500,000	3,000,000	20,000,000
Insight Communications		600,000		600,000
Verizon	7,000,000			7,000,000
Qwest	2,100,000			2,100,000
Media com		580,000		580,000
Cincinnati Bell	400,000			400,000
Local Net		260,000		260,000
Wind stream		400,000		400,000
Others			30,000,000	30,000,000
Totals	26,375,000	35,840,000	39,825,000	102,040,000
	Broadband Total	62,215,000

I believe that this table has the most accurate figures as regards DSL and Cable Access for each of the ISP or Cable Companies detailed in the Table at this point in time. I have not been able to ascertain any accurate data as regards dial-up access which remains surprisingly strong in the US. However, should anyone have any additional data or up to date, please let me know. I will add additional data for the UK and Europe as I can locate them out from published and unpublished sources.

The figure of 62,215,000 Broadband access either via DSL or Cable indicates around 8.7% growth over the figures published by the OECD in December 2006 which seems to be a reasonable assumption. I have included other forms of access either via Satellite or FTTH (Fibre-to-the-home) or FTTB (Fibre-to-the-building). I expect FTTH and FTTB to have grown significantly during 2007, but I haven't any specific data as such.

Direct Internet Navigation and Error Correction

Direct Navigation and error correction are becoming significant areas of web development, enhancing the Internet surfing experience for users, enabling ISPs to monetise hitherto error traffic, helping brands to grow traffic through keywords and building advertising revenues for PPC and CPA players through the delivery of qualified direct navigation traffic. Today the PPC sponsored click through or the pay for transaction approach is the fiscal model behind all of the current error and keyword redirection companies such as Golog, Barefruit and Paxfire.

Originally pioneered by RealNames in 1998, the user entered a keyword and was taken directly via a paid-for route to a specific web page. Hence ‘jobs’ would be bought by Jobs.com for $1m. Each time the word ‘jobs’ was typed in the browser address bar of IE4, the user was taken to jobs.com website via an interstitial advising the user that they were navigating by a RealNames keyword. RealNames had a database of over 1.5m keywords, all of which were paid for. They were both brand names and a limited number of paid-for generic terms. The user of generics was restricted by MSN to use by them to navigate to MSN results pages. Other examples of direct navigation technology existed including the AOL keywords and Common Names.

The raison d’être of direct navigation was that a study by MIT in 2000 revealed that each time a user was forced to click to the next page to uncover a particular location, 50% of the audience was lost. Hence if a deep site location was 5 clicks away for every 1000 users searching for the initial site only 31 users would follow through. A subsequent study by the University of Boston proved that direct navigation improved eCommerce conversion by between 65% and 300% depending on category .

RealNames was forced into Chapter 7 by MSN because they viewed the rapid growth of keyword use as damaging their search business. Between October 2001 and May 2002 RealNames PPVs rose from 2m per month to 30m per month. The growth was 75% per month.

The direct navigation approach used by RealNames was successful because it had near 100% of user coverage through the embedding with Microsoft IE3 and IE4. This was also its main Achilles heel; since MSN were able to terminate this exclusive contract.

Since 2002 there have been a number of direct navigation solutions based on keywords that have been deployed. Most notably AOL continues with AOL keywords (although their use is restricted AOL subscribers). Yahoo! has deployed their own version on Yahoo! enabled browsers, as have Firefox and Opera.

Two years ago Google moved to introduce a ‘paid-for’ keyword direct navigation system based on the RealNames approach with the so-called “Browse by Name” approach, as well as a browser based spelling correction service using stored URL data. Google offered both Brands and Generics with direct navigation. Since it used the Google enable browser they have dispensed with any interstitial page. Type “Ford Explorer” in Google Browser and it will navigate directly to the deep site Ford Explorer page. Similarly type “Mortgages” it will directly navigate to “UK Mortgages Online”. All these solutions are browser-based. Google moved away from monetising error traffic but maintained its direct navigation approach with it's Google Toolbar and through a commercial relationship with Mozilla to apply direct navigation to results and brand sites to browser bar search on Firefox. The only other major keyword driven system was deployed by J-Word in Japan, which is based on 3721 company's downloadable system. 3721, a Chinese company, was acquired by Yahoo! in 2003 for $120m.

Error correction as a result of mis-spelt domain names and broken links in search results, resulting from 400 and 500 class errors are relatively new phenomena. Initial approaches that have been adopted were limited to the analysis of non existent domains (NXDomain). The first to be introduced was SiteFinder from Verisign using “wild card” technology provided by Paxfire in the US. This technology identified when a non-existent domain was being sought and offered the user the opportunity to buy the erroneous input. However, the service responded to both the “A” record domain name request to the authoritative name servers and “Mx” record which is associated with email. Providing a positive answer to a incorrect email address resulted in opening ISPs SMTP mail systems to virus attack. The service was quickly abandoned after a formal request by ICANN , the Internet’s governing body. This experience has made ISPs very suspicious of any DNS “modifications”. However, the level of NXDomain traffic has been identified as being a significant source of error traffic. Total network error traffic is estimated by Verisign to account for approximately 15% of total network traffic of which NXDomain traffic makes up about 20%. A recent study by Google recently indicated that 404 HTTP errors alone account for 6.96% of published pages on average of the 11.5 billion web pages currently published on the web or some 800 million error pages!

A more sophisticated approach is being rolled out by a British company called Barefruit. Barefruit is aiming to provide a relevant result for nearly 100% of network error traffic by not only dealing with the mis-spelt URL error resulting in an NXDomain response, but also covering the 400 and 500 class errors.

The Barefruit Service
Barefruit has a service which is based on the evaluation of broken links in search results and the identification of DNS errors. It can resolve 400 and 500 errors in particular the eponymous 404 Error as well as isolate NXDomain errors. It does this by utilising a proxy server to store a temporary cache of the page prior to the display of the error page. It then detects the error response and then analyses the broken link data and provides a further set of links rather than an error page. Barefruit estimate that a typical broadband user will generate fifty errors every month.

Barefruit patented solution covers the broadest range of error traffic.

1 Mis-spelled urls (NXDomain)
2 Keywords
3 400 Errors
4 500 Errors

The addition, of direct navigation methodology is relatively simple to achieve by combining the NXDomain redirection patches currently in the public domain with an extended taxonomy.

It differs from the previous direct navigation approaches in that it works at the ISP level and not the browser level, although a browser download is available, only deploying when an NXDomain event occurs in the browser or when a 400 or 500 error occurs in a link.

The Barefruit solution provides the ISP with an opportunity to take back a degree of control over “their” user traffic rather than losing it to third parties and also creates a revenue stream from users clicking through on sponsored links. One significant fillup for Barefruit is the rapid adoption of DPI - Deep Packet Inspection, which allows broadband and cable operators to monitor their web traffic and so identify http errors relatively easily and allow Barefruit to analyse and identify user preferences and hence present highly relevant results instead of errors.

Other Developments

Other developments in ‘Direct Navigation and Error correction’ are being pursued by a number of players although none currently offer a solution as comprehensive as Barefruit.

Google has launched its own keyword system in ‘Beta’ which is a direct click through system running from the address bar of Google enabled browsers. They are also introducing spelling correction at the browser bar level for mis-spelled URLs of the URL data stored in their database. This operates in a predictive mode similar to the methodology used in mobile phone text services. It does not eliminate mis-spellings but does reduce MURL events. The approach is not DNS based but its deployment in Europe is limited by the need to comply with EU Directive EU 2002/53. It also tends to offer the ISP with limited income opportunities.

Yahoo! is also developing a direct navigation service similar to Google’s ‘I feel lucky’ approach, taking users directly to the sponsored website. This is likely to be deployed through Yahoo! enabled browsers and their ISP partners such as BT Yahoo! in the UK. BT Yahoo! enable browsers already use keyword navigation utilising direct navigation. Yahoo! will use a variation in their T&Cs to overcome the issues associated with the EU Directive.

Both Yahoo! and Google suffer from lack of independence in respect of ISPs given Overture’s ownership by Yahoo! and relationship with BT Yahoo! and Google’s recent link up with AOL.

Paxfire, a US based company, has been promoting its mis-spelled URL service to ISPs. The service is based on a proxy server solution and relies on a data update solution similar to the ‘black listing’ of sites deployed by website filtering solutions. It was this service which was the basis of the Verisign SiteFinder service and has been demonstrated to be difficult to configure successfully to avoid key issues. Paxfire's Look-Up Service allows ISPs to capture search terms and misspelled URLs entered into any Internet web browser address bar and deliver relevant content and advertising. Paxfire's technology works at the network level and provides a plug and play platform that requires installation of the Paxfire Look-Up Engine (PLE) in front of a DNS server in the ISP’s network.

Golog is limited to NXDomain events and the current business model in Europe and appears to rely on a “forced redirection” after 4 to 7 seconds. Golog installs a software patch into an ISPs DNS infrastructure which redirects the NXDomain to a Golog analysis server which identifies a key term which is then looked-up against a Golog database and where appropriate Golog monetises its mis-spelled URLs through partnership with Miva (formerly eSpotting)or a PPA partner such as Zanox. The Golog solution does not cache the request and cannot deal with any 400 or 500 error traffic. There are some risks associated with the “forced rediection” approach as it can result in the user feeling that they have received an unrelated result which is a poor user experience and can result in the advertiser being charged for a click that a user hasn’t selected.

Conclusion

The development and interest in direct navigation and error correction is growing at a significant rate. In the competitive on-line advertising market, its user ability and untapped potential, it has the potential to supersede the current listings and search approach as users become more familiar with the ability to use the address bar as a direct navigation tool combined with auto error correction both at the browser and in the search listings. One key aspect of any solution that an ISP may apply lies in offering its customers a route to "opt-out" of any error redirection.

To date only Barefruit has developed a combination of direct navigation and error correction which covers nearly 100% of error traffic and it also enables the ISPs to be offered a very powerful revenue generating service from what had previously been regarded as rubbish traffic.

Spam Phone Calls

As a marketeer I have always consider legitimate direct mail and phone canvassing as reasonable sales channels. However, I have now become so totally enraged with the endless number of spam telephone calls that I receive. The worst offenders appear to be Telephone companies who use Indian call centres to call me up to encourgage my adoption of say Toucan Telecom! They invariably introduce themselves as "Richard" or "Dave" when they are clearly Deepak or Sanjit and go on to tell me how much I could save if I change from BT to Toucan. The only problem is that their records are vastly out of date as I have been a customer of Sky for over a year.

The telephone sale is the scourge of modern life. I typically receive four calls each evening from Telcos sales, New Kitchens and Competitions for Timeshares. I have a similar number of "dead ender" calls which have nobody on the end of the call generator. I seldom let these callers get beyond their intro nowadays although my Partner is much more inclined to listen to them as she has no wish to be rude by slamming down the phone and feels sorry for the operators. Whilst I have some sympathy for the operators I have finally found a method which I hope will solve the problem. I have registered my telephone number with TPS (Telephone Preference Service) www.tpsonline.org.uk which aims to stop telephone marketing operations from abusing my telephone service.

I hope that this will help reduce the level of abuse. It will not stop those that come from the so called "competition operations". These I will report to OFCOM in the hope that they will take action, however, since I was the victim of BT's fraudulent downloading dialer scheme I doubt they will do anything! That is a story for another occassion! So if you are hacked off by repetitive marketing calls, start the fight back and use TPS!

You can also sign up to the Mail Preference Service on the TPS site and save a few trees from getting the chop!

Deep Packet Inspection - Is Big Brother here to help?

Deep packet inspection (DPI) is a form of computer network packet filtering that examines the data part of a through-passing packet, searching for non-protocol compliance or predefined criteria to decide if the packet can pass. This is in contrast to shallow packet inspection, which is usually called just packet inspection, just checks the header portion of a packet. This form of filtering is enormously powerful and will enable a huge range of new value added services such as the Barefruit http error solution I mentioned in a previous articles and will enable significant new revenue streams.

From a technical perspective, DPI devices have the ability to look at Layer 2 through Layer 7 of the OSI model. This includes headers and data protocol structures. The DPI will identify and classify the traffic based on a signature database and will allow the user to perform many things.

A classified packet can be redirected, marked/tagged, blocked, rate limited, and of course, reported to a reporting agent in the network. In this way, HTTP errors of different classifications may be identified and forwarded to Barefruit for analysis. Many DPI devices also perform the ability to identify flows rather than a packet by packet analysis.

DPI allows ISPs and cable companies to "readily know the packets of information you are receiving online ranging from e-mail, to websites, to sharing of music, video and software downloads" in the same way as a network analysis tool. This is the approach that Cable Operators and ISPs use to dynamically allocate bandwidth according to traffic that is passing through their networks. Thus a higher priority can be allocated to a VoIP call versus web browsing.

DPI is also increasingly being used in security devices to analyze flows, compare them against policy, and then treat the traffic appropriately (i.e., block, allow, rate limit, tag for priority, mirror to another device for more analysis or reporting). Since the DPI device looks at each individual packed, it can be used by ISPs to provide or block services on a user by user basis.

The continued growth in peer-to-peer (P2P) traffic presents increasing problems for broadband service providers. Internet service providers (ISPs) do not generate any revenue from delivering P2P traffic to their subscribers, and smaller ISPs face considerable peering costs when P2P traffic goes off-net. Even for ISPs large enough to cover these costs, P2P drives increasing traffic loads, requiring additional capex for no additional revenue. Moreover, a minority of users generating large quantities of P2P traffic can degrade performance for the majority of broadband subscribers using less-intensive applications such as email or Web browsing. Poor network performance increases customer churn, leading to a decline in service revenues.

Deep packet inspection (DPI) technology has emerged from the enterprise world into service provider networks to help mitigate the impact of P2P. According to most vendors, initial uptake of DPI was fastest in Asia, where problems with P2P traffic and high off-net traffic had been most severe. European broadband providers were also early adopters of DPI, but for another reason: Due to high levels of competition from digital subscriber line (DSL) broadband operators in many countries, service providers used DPI as a means to implement tiered service plans, to differentiate them from standard “all-you-can-eat" or "one-size-fits-all” data services.

In the U.S. market, multiple system operators (MSOs) such as Cable Operators were early adopters of the technology. This is because Cable Operators faced greater challenges than DSL providers in the last mile. For a Cable Operator, the last-mile bandwidth is shared among users, whereas in a DSL network a dedicated link is established for each subscriber. Smaller DSL operators were generally early adopters of DPI, as they suffered most from P2P-generated off-net traffic and peering costs. Recently, vendors note an increase in the level of request for proposal activity from large wireline and wireless operators in the U.S. It appears that several operators are looking to deploy DPI alongside their IPTV deployments in 2007.

Worldwide, network operators spent US$96.8 million (£48.4 million) on DPI in 2005, but the DPI sector grew by more than 75% in 2006, to about $170 million (£85 million) , and is forecast to exceed $586 million (£293 million) in 2010.

This may be one case where "Big Brother" really is here to help!