How ‘Big Data’ Can Help Build Better Cities

The Modi administration is expected to make big ticket investments in urban transportation projects. These projects include the ambitious 100 smart cities initiative, and several urban expressways and mass rapid transit projects.

The amount of investment needed for these projects is mind boggling and the exchequer isn’t exactly overflowing with surplus cash to fund them. Needless to say, all investments will have to be vetted carefully. And what better to invest in than robust data.

Traditionally, data required to plan and execute transportation projects have been collected manually through roadside and home interviews. As one can expect, such methods come with a wide range of issues ranging from reliability, inconsistencies, errors, inadequate sample sizes and high costs. Needless to say they are also open to manipulation. This is where Big Data could bring in transparency and greater efficiency in the delivery of transportation services.

What is Big Data?

It’s the name given to a large universe of tapped and untapped existing data on choices that people make over time and space. One could also call it ‘crowd-sourced’ data without actively involving the crowd. It could be an aggregation of online shopping trends, routes that people choose in a city, the speeds at which people drive along a corridor, or just a matrix of where people live and work/shop.

Big Data could be used in a large number of fields such as healthcare, governance, security, and market studies.  This article however is limited to the use of Big Data in delivering transportation services.

How is Big Data generated?

As far as urban transportation is concerned, Big Data mainly comes from four sources: Cell Phones, Bluetooth, Wi-Fi, and GPS devices.

Each time a user makes a call or sends a text message, his/her phone sends and receives signals from the nearest cell tower which gets recorded in the service providers’ database. The information that gets recorded includes (but is not limited to) call location and time of call.

These two parameters, aggregated over tens of thousands of call records spread over time and space, when plotted on a map, give planners invaluable information in making better informed decisions, much better than decisions based solely on traditional data sources.

Bluetooth and Wi-Fi work in a very similar manner. Data aggregators and public agencies install detectors across a road network for several days at a time. Each time a Bluetooth or Wi-Fi device passes these detectors, it’s MAC ID (the device’s unique identification number) and time of crossing gets recorded in a database. Analysts can then string together these timestamps and run queries to estimate travel time and speeds for each device (in a vehicle) recorded by the Bluetooth/Wi-Fi detectors.

The third major source are GPS logs from freight and taxi companies, public transport agencies, or even directly from GPS service providers such as Tom Tom and Navteq. GPS data is by far the best of the lot since it is continuous whereas the others are discrete in that they are available only at certain intervals of space and time.

For instance, let’s assume that the red circles in the map shown above represent cell phone towers in the Hyderabad metropolitan area. In this case, cell phone data logs for a journey from Secunderabad (top right) to Jyothi Nagar in Hyderabad (bottom left) will be available only at three location, the three red circles.

A GPS data log on the other hand could provide location and speed information for every tenth of a second for the entire journey. The problem with GPS though is that not everyone uses it and relying too heavily on GPS introduces a bias towards one set of users. The general practice therefore is to use a mix of all available sources validated with on-ground travel surveys.

How do end users obtain or collect Big Data?

Cell phone and GPS service providers, and freight, taxi and public transport companies are increasingly willing to sell this information to data aggregators such as INRIX. Companies like INRIX then collate this data and convert it into a more usable form before selling it to end users in the form of aggregated data sets.

The Big Question, what about privacy?

Electronic devices like cell phones have unique names, known as MAC IDs (no two devices can have the same MAC ID). It is nearly impossible to trace a MAC ID back to its owner without the manufacturer conspiring in this effort. The second level of security comes through data collection devices that intentionally encrypt unique MAC IDs to produce a scrambled unique ID that cannot be traced back to the original MAC ID (or the owner).

All that is captured in the dataset is a log of anonymous IDs with corresponding timestamps and geographical coordinates. The third level of security comes when Big Data vendors aggregate tens of thousands of records to convert the raw data into more useable formats and fancy maps.

So what exactly can we use Big Data for?

As stated earlier, Big Data’s utility is limited by the user’s imagination. As far as delivery of transportation services is concerned, Big Data could be used in a wide range of ways such as:

1.Building citywide land-use and transportation planning models

2.Identifying Metro alignments and bus routes

3.Identifying accident hot-spots and bottleneck locations

4.Traffic management schemes for special events and construction periods

5.Intelligent Transportation Systems (ITS) such as dynamic message signs

6.Before and After studies to evaluate the efficacy of completed projects

7.R&D

For instance, the hypothetical travel demand heat map below shows Cell Phone data could be used to identify the busiest travel corridors in a City, which in turn could help planners plan roads and mass transit services.  As stated previously, such plots are based on several days’ worth of data collected from hundreds of thousands of users and hence provide far more accurate information than any of the traditional methods ever could.

At best, traditional methods would give information for a day’s worth of travel activity, nowhere near Big Data’s sample size.

Big Data could also be used to evaluate previous schemes implemented in a city. For example, how effective was Hyderabad’s Outer Ring Road in mitigating congestion could be determined simply by comparing cell phone data from years preceding the construction of this road with cell phone data for years during which the road has been in operation.

This sort evaluation can only be done with Big Data as it allows agencies to go back in time and see how their projects have performed over the years. For a developing nation like ours, every lesson learnt is worth its weight in gold as it could save us millions on future transportation projects.

Intelligent Transportation Systems (ITS) are going to be a key component in the 100 smart cities initiatives. Among other things, ITS includes what are known as dynamic (or variable) message signs (VMS).  VMS are digital signs that are used to warn motorists of accidents and congestion ahead, or just provide real time information such as travel speeds on a corridor to help drivers make the best decision about their travel routes.

For instance, in the hypothetical congestion scenario shown in the map below, armed with Big Data and VMS, the Hyderabad Metropolitan Development Authority’s (HMDA) traffic management centre could send a message to their VMS warning motorists of a problem (shown in red) along Route 1 and advise drivers to use Route 2 (or Routes 3, 4 and 5) instead.

This strategy not only results in better utilisation of roadway capacity but also improves travel time reliability for motorists on the road. It is a bit like having your friendly traffic policeman clearing traffic for you, the only difference is, this time it’s based on robust data and not on his whims.

What should the Government of India do?

Innovation generally comes in from the private sector but things gets complicated in a country like ours. The private sector is ready for Big Data but is unlikely to invest in it fearing the dinosaurs in our public offices. The push for such things therefore must come from the Central Government.

The Ministry of Urban Development has taken some positive steps such as publishing a paper on Smart Cities. The Smart Cities paper is exactly the kind of document to tell the industry that the GoI wants cutting edge technologies such as Big Data enabled solutions for its Smart Cities.

As a second step, the Government should nudge Delhi Metro Rail Corporation (DMRC) and its subsidiaries to evaluate Big Data on all its future projects. This will not only help DMRC but will hopefully have a trickle-down effect on municipalities, public transport companies, and metropolitan development authorities that work closely with the DMRC.

With the rapid technological strides being made in the automobile industry, Big Data will have an additional role to play, that of an enabler for ‘connected’ and ‘autonomous’ vehicles of the future. ‘Autonomous’ or self-driving vehicles are likely to hit the road in the next couple of years but futurists and planners expect that these vehicles will be the norm (at least in the west) by 2040.

To realise their true potential, such vehicles will need a lot of real time data such as speed limits, signal indications, congestion, weather forecast, and even the ability to ‘talk‘ to adjacent vehicles and objects. Big Data therefore is not a matter of if, it’s just a matter of when. Sooner the better in my opinion.