The Future of the Real-Time Web – Part 2

This is the second part of a four part blog post series about the future of the real-time web.

Real-time Information: Importance of Filters and Agents in Making Use of Available Information
The internet has been a great promoter of data generation, with estimates of a petabyte of data being generated daily (that’s 1 million gigabyte or about ten thousand large hard drives each day)! The great thing about the internet is that data is not only generated by professionals or institutions as is common with traditional media (think of TV stations, publishing houses, etc.), but also comes from a large amount of individuals. The growing importance of networks such as Twitter and Facebook, for example, has led to Google indexing user-generated content in its search-engine.

Data on its own is abundant and can exist in many forms. Every second, new data is released on the web in real-time and becomes accessible immediately. The data becomes information after it has been given meaning by way of relational connection. So identifying and combining relevant data becomes the challenge. Software solutions such as the open source Hadoop project help in processing large amounts of data. Today, filters and agents provide users with information relevant to them. Most websites today are accessed via search engines that allow users to identify relevant information through search terms. And since there is so much information out there on so many websites, those that maximize usability and user experience will likely be winners in user retention – the real capital of many websites. Irrelevant or hard to retrieve information will only serve to repel users from web services. With relevance being key, suppliers of intelligent systems will continue to be heavyweights of knowledge. One should look no further than Google or Wikipedia as examples of successful agents and filters. Other important information services include RSS feeds and other means of staying up-to-date on new information as it comes online – in real-time.