Project Coordinator (EU) :Tracking Exposed
Country of the EU Coordinator :France
Organisation Type :Non-profit Organisation
Project participants :
EU Team - Tracking.Exposed:
Marc Faddoul is the Co-Director of Tracking Exposed, AI Forensics, he is the project leader responsible for coordinating the overall scope and outcomes of the project.
Salvatore Romano is the Head of Research at Tracking Exposed, AI Forensics and he is responsible for coordinating the research, experiments and report writing.
Ilir Rama is a Post-Doctoral researcher at the University of Milan and researcher at Tracking Exposed, AI Forensics. He works on data collection on the web app and the methodological aspects of the report.
Giulia Giorgi is a Post-Doctoral researcher at the University of Milan and researcher at Tracking Exposed, AI Forensics. She is responsible for the elaboration of the results and the conclusion of the report.
Gaetano Priori is the Mobile Application Analyst at Tracking Exposed, AI Forensics and is dedicated to the data collection on mobile.
Justin Yeung is a researcher at Tracking Exposed, AI Forensics, and works on the platform's features analysis. His research interest lies within communication in the digital world, tech regulations and philosophy and computational methods.
US Team - Algorithmic Transparency Institute:
Cameron Hickey is the Director at Algorithmic Transparency Institute and works on the data collection platform for mobile.
Athanasios Andreou is a researcher and developer at the Algorithmic Transparency Institute, and works on data collection and analysis on mobile.
State of US partner :Washington
Starting date :
The TikTok Observatory: Interconnecting Junkipedia and Tracking Exposed's monitoring infrastructures
Our overall goal is to create an open-source and accessible infrastructure to monitor the ongoing censorship trends on TikTok, and the possible algorithmic demotion/promotion related to censorship. We aim to emposer researchers through three main contributions.
- A detailed technical report considering the difference and similarities of TikTok's algorithmic systems between mobile and web;
- An open source and accessible infrastructure for the study of the platform, informed by the result of the technical report;
- A research protocol and methodology, informed by our findings and leveraging on the provided infrastructure.
More specifically, this experiment will try to definitely answer whether the browser application can be used as a proxy to investigate the platform as a whole, focusing on the different points of contact with the users, such as the For You Page, search results and audio recommendations.
This project has allowed Tracking Exposed and Algorithmic Transparency Institute, two leading non-profit civil society organisations tin the filed of algorithmic accountability, to set up a solid partnership to outlive the grant. In terms of the experiments impacts they are summarised below.
Impact 1: Enhanced EU – US cooperation in Next Generation Internet, including policy cooperation.
Since the analysed platform has a significant impact on the citizens of the WU and the USA, our research is relevant. It will be used to refine, investigate, litigate and advocate for change with cross-Atlantic potential. A cross application methodology is crucial for all future investigations on the platform by avoiding blind spots that could encourage algorithmic opacity and non-transparent policies.
Impact 2: Reinforced collaboration and increased synergies between the Next Generation Internet and the US Internet programmes.
The work contributes to the trustworthiness of the Internet, which is a specific objective of the NGI programme.
Impact 3: Developing interoperable solutions and joint demonstrators, contributions to standards.
We developed a protocol harmonising Guardoni and Junkipedia's infrastructures. Such a protocol allows both infrastructures to work in parallel, providing a methodology to analyse and compare the algorithmic personalisation of TikTok's "For You" page across different applications (i.e. mobile-based app and web-based app).
Additionally, this protocol addresses one of the most common problems in studying recommendation systems: the cold-start. Cold-start refers to a limit in an algorithmic recommendation system, where it struggles to provide relevant content due to a lack of information about the user. In the context of research and auditing, this translates into poor data collection due to usage patterns far from that of an average user.
To address these problems, we provide documentation allowing replication of our research protocol, which leverages the Guardoni software, a free and open source tool now enriched with new data-collection and browser orchestration features that allow for cross-application comparison.
Impact 4: An EU - US ecosystem of top researchers, hi-tech start-ups / SMEs and Internet-related communities collaborating on the evolution of the Internet
The unprecedented knowledge share among the EU and USA organisations working independently on TikTok's algorithm audit is crucial to developing a new ecosystem of top researchers and non-profit organisations. Among the few actors involved in this type of analysis, there is rarely enough time and resources to establish a common standard or a cross -organisation development plan. On the other hand, platforms like TikTok have a global audience and a massive amount of resources to adjust their algorithm to follow their interest. Moreover, since platforms implement only some of the policies equally across the world, cross-national analysis allows research groups to exploit regional regulations and vulnerabilities in a worldwide effort to push for algorithmic accountability.
We are making special efforts to disseminate out results to the other actors in the ecosystem, through participation in workshops and conferences. In particular, we participated in the DigiMeth festival hosted by the Centre of Interdisciplinary Methodologies as the Warwick University in the UK and the Digital Methods Winter School 2023 at Amsterdam University, which are two of the most relevant European hubs for Media Studies academic researchers. Additionally, we will participate in the next Mozilla Festival 2023, a pivotal moment in the yearly discussion around research, open software and digital rights.
The expected results of the experiment is to understand whether the algorithms on web and mobile TikTok work differently than each other. With our approach, we expect to collect robust evidence to examine if they significantly differ from each other. If they do differ from each other, future research require two sets of tools to retrieve and collect data from the web-based and mobile TikTok. In the case of insignificant differences, based on our current technologies and techniques, web tech will be the better choice for further research, as it is more stable, independent and resilient.
Future Plan :
Future research should expand on the focus of these findings by adopting various devices and real use-cases. This means incrementing the number of use-cases tested, for example, by including iOS versions of the App; different countries or devices; and deploying experimental research designs involving real users with differing browsing patterns. Such an approach would further narrow the scope of these findings, paving the way for implementing a state-of-the-art data collection architecture based on a combination of natural or emulated mobile devices.