What Ment.io’s Thought Distance Analysis Teaches Us About How Activists and Bots are Attempting to Sway American Discourse
At Ment.io, our algorithms determine the polarization, thought distance and other behavior patterns of online conversation. We’ve recently been doing research which allowed us to determine the following about Twitter discourse regarding the ongoing war in Ukraine:
- How pro-Ukraine bots are strategically targeting the most influenceable US politicians.
- Distinct behavior differences between bots and activists at scale.
- Trends of how this and other controversial discussions become more or less polarizing over time.
Thought Distance and the Twitter Battleground
The battleground of the Russia-Ukraine war is not only fought on the ground and in the cities, but also in the virtual world of social media, and Twitter in particular. Analyzing the large volumes of tweets that accompany such a controversial event is still considered a significant challenge with existing technologies.
With Ment.io’s proprietary social media analytics platform and algorithms we are able to gain insights into the behavior of activists and bots, in order to show how a pro-Ukraine cyber campaign is trying to sway American policy by targeting authoritative figures from a specific segment of the political spectrum. The core technology relies on AI-based stance detection and thought-space mapping using the Thought Distance metric.
Stance Detection is a known NLP (Natural Language Processing) challenge. At Ment.io, we generate best-in-class stance detection AI models that classify posts to supporting, opposing, and neutral categories, based on a high quality proprietary dataset.
Thought Distance is a metric that builds upon our stance detection classification. Thought distance measures the level of agreement/disagreement between two actors that participate in shared conversations. This metric is particularly useful when analyzing social media conversations on contentious or polarized topics.
When combining thought distance between many actors, a Thought Space map emerges, where like-minded individuals coalesce to clearly identifiable clusters. Analyzing this thought space can yield important insights into the ongoings on social networks.
This report focuses on the Russia-Ukraine war, yet similar analysis is ongoing for the COVID-19 vaccines opinion war and for the January 6th US Capitol attack. This report summarizes initial findings as work continues on improving stance classification, automatic bot detection, increasing user reliability estimation, improving stance cluster detection and visualizations.
Partisanship Detection with Thought Distance
The Thought Space map below is made up of nodes and edges. Nodes represent users and edges represent the thought distance metric between them, aggregated over all their mutual conversations and calculated using the underlying stance detection AI classification. The closer two individuals are in terms of their thought distance, the larger the pull between them.
Figure 1 – Partisanship Clustering Using Thought Distance
A clear partisan division emerges in the map. Colored nodes represent US politicians and include all congress members that have expressed their opinion about the war on Twitter. Republicans are red and Democrats are blue. Edges are colored based on the thought distance relationships, where a green connection means the two users think alike, and a magenta connection means they oppose each other’s views.
Both politicians and users are clearly clustered by partisan perspective in the figure above. Politicians are more spread out, mainly because they don’t reply in conversations, but rather broadcast tweets that become the root of political conversations. The unique formation at the bottom is a set of pro-Ukrainian bots, covered in more detail later in the research.
Distinguishing Between Bots and Activists
Bots generate huge amounts of uniform tweets, but so do some activist groups, who often post the same message repeatedly in order to promote their agenda. It’s similar to the difference between telemarketing and spamming. The former is a bothersome yet legitimate advertisement, while the latter is considered unreasonable and often illegal.
With thought space maps the distinction between bots and activists becomes evident. In the map below the bot group is clearly isolated. These bots focus on the same conversations, expressing similar, often identical, views. They try to mimic human behavior, but ultimately create a distinct and repetitive echo-chamber. This behavior is common to a horde of bots, operated by the same individual or organization.
Figure 2 – Identifying Bots vs Activists Clusters With Thought Distance
Social activists of shared ideology also have many internal supportive interactions, but these are more unique and less repetitive. They also interact naturally and reciprocally with their thought space neighbors, keeping them embedded in the map.
|Group internal interactions||Significant and supportiveRepetitive||Significant and supportive|
|Group external interactions||LessUnreciprocated||More|
The following are message examples of bots as compared to activists with the number of times each profile repeated the same message (with different mentions of users) for comparison. You will notice how the activist messages are more unique and repeated less frequently.
Top Messages From Bots
|@perfect2upset||172||CLOSE THE SKY OVER UKRAINE!|
|@innilovely||160||CLOSE THE SKY OVER UKRAINE!|
|@lilmeowlovely||116||CLOSE THE SKY OVER UKRAINE!|
|@vladshel||101||CLOSE THE SKY OVER UKRAINE!|
|@badzianaa||77||CLOSE THE SKY OVER UKRAINE!|
|@alzamesova||59||CLOSE THE SKY OVER UKRAINE!|
|@Viktori27947662||59||Close the sky! THERE IS NO OTHER WAY TO STOP THE WAR.Your Country will be the next!|
Top Messages From Activists
|@BogdanaMadar||17||Remember 9/11? Every day in #Ukraine is 9/11! Putin is bombing residential buildings, hospitals, kindergartens killing thousands of people. #ProtectUASky to stop terrorists!|
|@SaintMoonKyiv||16||Russian terrorists are killing not only Ukrainians but also Americans. How many more civilians are to die? Let’s stop evil together! We need your help!|
|@Stelmashow||12||#Putin’s bombing is already close to #NATO border – 20 km. And the US is still afraid of having #NATO involved in this war? Show your leadership! #|
|@mudrak2403||12||#Putin’s bombing is already close to #NATO border – 20 km. And the US is still afraid of having #NATO involved in this war? Show your leadership! #ProtectUASky|
|@JackShendrikov||12||Russian military shot dead US citizen, journalist Brent Renaud in #Ukraine! Isn’t it enough for finally taking actions and #ProtectUASky? Make your president walk his talk. Biden promised to respond forcefully if #Putin attacks Americans in Ukraine.|
|@ikotlyarenko711||12||Russian military shot dead US citizen, journalist Brent Renaud in #Ukraine! Isn’t it enough for finally taking actions and #|
Targeting Specific Politicians with Pressure Campaigns
The bots and the activists are not engaging with politicians at random. Naturally, politicians are on the outskirts of the thought space map, as they tend to post tweets that later get many responses, rather than responding themselves to others’ tweets. If we compensate for this effect with clustering algorithms and embed the politicians in the center of their thought cluster in the map, we can more easily see attempts of activists and bots to influence the politicians.
Figure 3 – Most Targeted Politicians by Activists
The identified pro-Ukrainian activists group is heavily engaged with a specific subset of politicians who are mostly Democrats, as well as a few Republicans. Both these politician groups belong to the center left of the political spectrum, even though they are split between the political parties.
Most of the Democrats in this cluster, such as Josh Harder (D-CA 10th District), Juan Vargas (D-CA 51st District), Mike Quigley (D-IL 5th District), Brendan Boyle (D-PA 2nd District), Raul Ruiz (D-CA 36th District) and Suzan DelBene (D-WA 1st District) are members of the New Democrat Coalition and have relatively conservative fiscal agendas. Respectively, the Republicans in this cluster tend to be more on the liberal wing of the Republican party and come from swing states. It makes sense for those who want to make the most impact to target these specific sets of politicians and add pressure to increase involvement in the Ukraine war.
The Future of Thought Distance on Social Media
The Thought Distance metric and thought-space graphs can be used in a variety of use-cases, such as:
- Identifying partisan clusters of users and tweets
- Identifying bots and distinguishing between activists and bots
- Identifying pressure campaigns in social media
The Thought Distance metric is based on Ment.io’s proprietary AI stance detection platform. It is a unique signal that can be complementary and often have advantages over traditional signals such as interaction counts and sentiment analysis.
For the specific case of the Russia-Ukraine war, we were able to identify an active bot cluster running a pressure campaign targeting center-oriented politicians from both major parties. We were also able to identify and separate the top activists in this space.
Future work will improve over the techniques presented here and include tooling and automation to allow such analysis in real-time as events unfold. Other areas we are actively looking at are COVID-19 vaccines and the January 6 Capitol attack.