In our digital world, data is everywhere. It’s estimated that as a global community we create around 2.5 quintillion bytes of data each day – that’s 2.5 followed by a staggering 18 zeros.
Much of this data can be classed as alternative data, also known as external data or simply ‘alt-data’. It’s termed alternative because it differs from the ‘traditional’ types of internal data that organisations have.
Alt data is created by individuals in the form of public social media posts, but also by companies and organisations in outputs like press statements and job vacancies. It can also take the form of data from sensors and devices, which provide openly available feeds of geospatial data – for example flight trackers and live car parking space updates.
All our daily digital activities leave a lot of unstructured, messy public data in their wake. But if we can gather this efficiently and responsibly and knock it into shape, it can provide predictive signals and valuable insights – which can help us make better decisions no matter whether we are a business, a charity or a public body.
Alt data is already heavily used by the financial sector for investment decisions. But it also has the exciting potential to be used in many other ways, including tackling the climate crisis, improving public safety and supporting people to have better lives.
As outlined in its National Data Strategy, the UK Government sees the opportunity to unlock the potential of data to create a ‘fairer, better society for everyone’. At Bright Data, that’s an ambition we share – and we have many ideas about how alt data can play its part.
In studies academics have been able to make accurate estimates of infection rates by analysing the content of public Tweets – making alt-data a potentially powerful tool during a health emergency. During Covid-19 we saw a blend of data being used to identify vulnerable people needing food deliveries. Building on this data success story, we could access and analyse masses of publicly-available social media posts and use them to identify specific geographic areas where harmful misinformation is spreading. Cost-efficient, targeted digital communications could then be run in these identified hotspots.
You may have heard about an exciting new climate change ‘accelerator’ called Subak, which aims to use data to supercharge efforts to tackle global warming and climate change. Experts have already helped turn satellite data into cloud cover forecasts, to predict solar power output and reduce emissions from other power sources – and this is just the start. We could do things like pull together external data on meat prices from disparate sources, giving better insight on changing demands and consumer habits, helping reduce waste and over-production. Effective use of alt data from public social media posts could also provide the electric car industry and public authorities with a richer picture on take-up rates for new green vehicles – and the unmet needs holding back greater adoption.
Official unemployment figures are an important source of data for those running any country, city or town. But as the US-based Economic Policy Institute puts it, “in a complex economy, conventional measures sometimes fall short”. Automated web data tools that can retrieve tens of thousands of public online job postings could be used by national and regional governments to track the labour market during an economic recovery. The insight this type of alt data reveals could be used by authorities and providers to provide more relevant, specific and timely support and training to jobseekers, as well as those who are in work but seek better jobs
Sadly, during the first Covid-19 lockdown the number of reported incidents of children suffering abuse or neglect in England rose by a quarter. As highlighted by the BBC, the pandemic meant children were often hidden from view of professionals who are best placed to spot signs of abuse. We know from our work with Israel charity ELEM that when young people are being abused they can often leave digital footprints on public channels in order to be traced. With the help of our technology, ELEM was able to tap into massive amounts of openly available social media data to feed a machine learning system that helped identify young people who might be being abused. Alt data could be used in a similar way to help charities find children and young people in the UK who need support on everything from abuse and neglect, to mental health, to eating disorders.
The amount of data we create will, undoubtedly, continue to grow. The technology and tools are here, right now, to access lots of external data from many different open public sources. But we must do this while being compliant and responsible.
Research from the Open Data Institute suggests that people may be happy with their data being used to benefit society, but may not want that data being used to assist the investment decisions of hedge funds. An ethical or fair-by-design approach to accessing data is key to our approach and we are committed to being part of a thriving, responsible global data industry.
Transparency and openness are key if we are to build a level of public understanding and trust that allows the world-changing, future-shaping potential of alt data to be truly realised.