In recent months, a successful pilot was conducted at the Dutch news website NU.nl of a system for automated news detection on Twitter. The results showed that the dynamic dashboard ‘RTreporter’ appears to add value. “The dashboard is useful for picking up news that quickly increases in virality,” says Mark Vos, editor social media at NU.nl. “It’s already given us an edge over other media on various occasions. Every day the dashboard provides us with multiple news items we otherwise wouldn’t have seen.”
Since Twitter started in 2006, the service has often proved its value to journalists. Time and again, the microblog platform has functioned as a source of tips and news. But uncovering newsworthy reports in the enormous number of tweets − 400 million tweets worldwide, some 6 million of which from the Netherlands − is a daunting task for newsrooms. Particularly when you don’t know what you’re looking for.
The Dutch company CCinq has been working with Amsterdam’s VU University for over a year on an algorithm that will enable the generation of automated news alerts. The project, which was supported by the Netherlands Press Fund, has now led to the first version of a dashboard, RTreporter. The system scans all Dutch-language tweets, seeking to discover rapidly emerging trends. By using intelligent clustering of words, tweets can provide an indication of incidents or breaking news.
“Its strength lies primarily in the local and direct signalling function: detecting the event while it’s unfolding,” says Mark Vos. “This means we can make an early assessment of the development of an incident and its value/newsworthiness for our newsroom. It should be noted, incidentally, that just a few reports from eyewitnesses would not create the critical mass needed to signal a full alert. This often requires a catalyst, such as a alert sent out by the police.”
While RTreporter clearly proved its value during the pilot, it is not a miracle worker. “Monitoring the dashboard is a permanent process that requires the journalist to constantly consider the newsworthiness of emerging trends,” Mark Vos explains.
In the NU.nl newsroom, the daily news is now compiled and reported with the help of RTreporter. As a result, the speed of news gathering is often faster compared with the established press services.
The journalists very much appreciate the transparency of the dashboard. “Thanks to the ability to click through to clusters, the sources can immediately be consulted and, with just a little practice, it’s quickly apparent if something is ‘unjustifiably’ included or that a development has real news potential,” according to Vos.
Another nice feature is the dashboard’s overview of photos tweeted during events, Vos says. “It helps our team when approaching eye witnesses and the addition of images improves our reporting.”
According to NU.nl, a significant advantage of RT is the lack of a ‘moderator’ between the events and those who witness them. “Journalists view the events via x-ray equipment, as it were, and are in an excellent position to add value by combining their own knowledge and skills with what has been witnessed,” says Vos. “As a result, we identify events sooner, such as streets that are evacuated in outlying areas. Ultimately, that brings NU.nl closer to people in the immediate vicinity.”
Various adjustments to the dashboard were made during the NU.nl pilot in an effort to align the application as closely as possible to the needs of the editorial team. For instance, the dashboard can now be used for sub-topics such as stock market reports and sports. A user-friendly backend is also in the works, enabling journalists to gear the dashboard more closely to their wishes. This will mean they can adjust the algorithm’s thresholds, among other things. And in future, users will be able to add their own glossaries, so Twitter can be scanned by sub-topic.
The NU.nl editorial team will continue using the dashboard for the next six months and their experiences will be applied to further improving the product.
Mark Vos is clearly pleased with RTreporter. “Its real advantage comes down to very regular monitoring and a knowledgeable team. Just like the emergency services, that team then rapidly scales up and makes the most of its expertise as a gatherer and provider of news. Important elements in this process include: a thorough knowledge of the Twitter landscape along with a sixth sense for hoaxes and insight into emergency services communication. RTreporter supplies the news items, which the editorial team then picks up and reports. We can now often do that more quickly than before.”
Twitter itself is also excited about the dashboard. According to Mark S. Luckie, Twitters’ Manager Journalism & Media and former Washington Post journalist, these are precisely the tools that Twitter is eager to see developed. Additional talks will be held with Twitter in the near future about collaborating on further developing the dashboard.
CCinq is now working on versions of RTreporter in other languages. English, German and French dashboards are currently being tested and results are promising. “We are very exited about the cooperation between the mathematicians of the VU University and journalists”, says Bert Kok of CCinq. “Science and journalism should work together more often on innovations like RTreporter”.