The TikTok Observatory: Interconnecting Junkipedia and Tracking Exposed's monitoring infrastructures

---

Impacts :

This project has allowed Tracking Exposed and Algorithmic Transparency Institute, two leading non-profit civil society organisations tin the filed of algorithmic accountability, to set up a solid partnership to outlive the grant.  In terms of the experiments impacts they are summarised below.

Impact 1: Enhanced EU – US cooperation in Next Generation Internet, including policy cooperation.
Since the analysed platform has a significant impact on the citizens of the WU and the USA, our research is relevant.  It will be used to refine, investigate, litigate and advocate for change with cross-Atlantic potential.  A cross application methodology is crucial for all future investigations on the platform by avoiding blind spots that could encourage algorithmic opacity and non-transparent policies.

Impact 2: Reinforced collaboration and increased synergies between the Next Generation Internet and the US Internet programmes.
The work contributes to the trustworthiness of the Internet, which is a specific objective of the NGI programme.

Impact 3: Developing interoperable solutions and joint demonstrators, contributions to standards.
We developed a protocol harmonising Guardoni and Junkipedia's infrastructures. Such a protocol allows both infrastructures to work in parallel, providing a methodology to analyse and compare the algorithmic personalisation of TikTok's "For You" page across different applications (i.e. mobile-based app and web-based app).

Additionally, this protocol addresses one of the most common problems in studying recommendation systems: the cold-start.  Cold-start refers to a limit in an algorithmic recommendation system, where it struggles to provide relevant content due to a lack of information about the user.  In the context of research and auditing, this translates into poor data collection due to usage patterns far from that of an average user.

To address these problems, we provide documentation allowing replication of our research protocol, which leverages the Guardoni software, a free and open source tool now enriched with new data-collection and browser orchestration features that allow for cross-application comparison.

Impact 4: An EU - US ecosystem of top researchers, hi-tech start-ups / SMEs and Internet-related communities collaborating on the evolution of the Internet
The unprecedented knowledge share among the EU and USA organisations working independently on TikTok's algorithm audit is crucial to developing a new ecosystem of top researchers and non-profit organisations.    Among the few actors involved in this type of analysis, there is rarely enough time and resources to establish  a common standard or a cross -organisation development plan.  On the other hand, platforms like TikTok have a global audience and a massive amount of resources to adjust their algorithm to follow their interest.  Moreover, since platforms implement only some of the policies equally across the world, cross-national analysis allows research groups to exploit regional regulations and vulnerabilities in a worldwide effort to push for algorithmic accountability.

We are making special efforts to disseminate out results to the other actors in the ecosystem, through participation in workshops and conferences.  In particular, we participated in the DigiMeth festival hosted by the Centre of Interdisciplinary Methodologies as the Warwick University in the UK and the Digital Methods Winter School 2023 at Amsterdam University, which are two of the most relevant European hubs for Media Studies academic researchers.  Additionally, we will participate in the next Mozilla Festival 2023, a pivotal moment in the yearly discussion around research, open software and digital rights.

Results :

The expected results of the experiment is to understand whether the algorithms on web and mobile TikTok work differently than each other. With our approach, we expect to collect robust evidence to examine if they significantly differ from each other. If they do differ from each other, future research require two sets of tools to retrieve and collect data from the web-based and mobile TikTok. In the case of insignificant differences, based on our current technologies and techniques, web tech will be the better choice for further research, as it is more stable, independent and resilient.

Future Plan :

Future research should expand on the focus of these findings by adopting various devices and real use-cases.  This means incrementing the number of use-cases tested, for example, by including iOS versions of the App; different countries or devices; and deploying experimental research designs involving real users with differing browsing patterns.  Such an approach would further narrow the scope of these findings, paving the way for implementing a state-of-the-art data collection architecture based on a combination of natural or emulated mobile devices.

NGI related Topic :

Experimental Platform interconnections

Call Reference :

5