Facebook Ads Auditor¶

In the following notebook, it is explored methodologies and limitations of using the Facebook Graph API to access the Facebook Ads library performing data analysis from an auditing viewpoint on the Dutch credit housing market getting key insights of advertisement practices.

The notebook is divided in 3 main sections:

• 1. Facebook Ads Library
• 2. Accessing the Facebook API as Developer
• 3. Discovering Key Insights Analysing Ads Data

The first section discusses how to use and access Facebook ads data introducing the Facebook Ads Library service. The second section shows how to connect directly to the Graph API in developer mode and gather advertisement data directly, and thus perform data analysis for auditing. Finally, the third section presents an auditing data analysis exploring some key insights that could be useful for Consumer Protection Investigations

Key Insights Summary¶

In this data auditing, advertisement data has been automatically collected on 500 ads using Facebook's Ad Library API. It is been searched the API by all advertisers and all active and innactive ads available. We have observed that the average time of ads impressions is 8 days to the consumers, particularly, the 45+ were the most microtargeted group with ads impressions for more than a week in most cases. Interestingly, there were certain ads that microtargeted very specific groups reaching a huge audience (more than a 1M impressions) in a short amount of time (less than a day), these came from CDA-jongeren, the International Literature Festival, and ASN Bank, targeting 18-24 male, 65+ female and 18-24 female respectively. Surprisingly, the ad related to the festival contains the keywords and together with the ASN ads are tagged as Issues, Elections or Politics which is odd but probably strategic. ASN Bank campaign also shows to be the longest one, this is interesting since there is no other bank are showing in the data. Furthermore, it is been audited the authenticity of links and the behaviour of campaigns and there is no indication of scammers in this data. The most microtargeted region was Paramaribo, with one campaign that was shown only there. But also we see cases where the demographics are not specified, therefore, groups of 13-17 are exposed. In conclusion, the fact that Facebook only allows accessing ads tagged as Issues, Elections or Politics creates a big limitation to audit, Because even though we can perform queries related to the house market we are only getting the tagged ones and we don't know how much are we missing.

Data acquisition and analysis were done using the Python programming language. Plotly for visualization and word frequencies by using the Natural Language Toolkit. The ads data and code used for the analysis can be found in the Github repo of this work. Any comments, please contact us at p.hernandezserrano@maastrichtuniversity.nl

Date: Sep 2021
Author: Pedro V Hernández Serrano
License: Attribution 4.0 International (CC BY 4.0)

1. Facebook Ads Library¶

The Facebook Ads Library is an open repository of all the ads and campaigns, active and inactive from many countries that Facebook runs. This repository is exposed as a service with a search engine interface that will help users to easily look up for ads, campaigns, Facebook pages, etc. in the Facebook ads database, this service can be accessed by using the Facebook Graph API. The search interface has two filters Country and Ad Type.

Limitations:

• The service is limited for the user to select one country at a time

• The service only allows to look up All ads at once or the Issues, Elections or Politics category.

  Unfortunately Facebook is not making public the rest of their defined categories, this should be followed up by data-related regulation conversations.

• The service shows the details of each ad, but actually only show the ad identifier, the link to the page, and the ad content, only the Issues, Elections or Politics ads contain information about the demographics of the audiences, the rest of the ads do NOT contain demographics

• Using the API is impossible to do the last query since Facebook is only making public the parameter POLITICAL_AND_ISSUE_ADS therefore the rest of the ads are not accessible via the API

The following EU technical in ads transparency report documents the use of Facebook Graph API and how was used for analysing general elections ads https://adtransparency.mozilla.org/eu/methods/

Approach:

The following auditing of ads will access all the ads tagged as POLITICAL_AND_ISSUE_ADS given that is the only option. In the following analysis, we perform a query on Hypotheek related topics getting a number of ads, however, we won't get the Hypotheek related ads that are not tagged as POLITICAL_AND_ISSUE_ADS and we will never know how many are we missing (only Facebook knows).

2. Accessing the Facebook API as Developer¶

The Facebook Graph API is the primary way for apps to read and write to the Facebook social graph. The official documentation is found in APIs and SDKs docs here. The Graph API has many uses, from creating and publish a game to analyze friends networks, naturally contains the Ads that Facebook publishes but only a limited amount of those, only limited to social issues and politics. In order to access and use the API, you need to gain access to the Facebook Ads Library API at https://www.facebook.com/ID and confirm your identity for running (or analysing) Ads About Social Issues, Elections or Politics, which involves receiving a letter with a code at your official account and sending picture identification to Facebook. Basically one has to be registered as an official Facebook developer, and this permission can actually take from one day to weeks.

The Facebook API has also a nice user interface (for developers) called the Graph API Explorer, which allows the developer or analyst to quickly generate access tokens, get code samples of the queries to run, or generate debug information to include in support requests. Here more info.

Requirements

1. Register as a developer at developers.facebook.com
2. Go to Graph API explorer and create an app
3. Having a new app ID. Create a Token for the new app in the UI
4. Define the Graph API Node to use: ads_archive

An example query that can be retrieved from the Graph API Explorer is the following

        ads_archive?access_token=[TOKEN]
        &ad_type=POLITICAL_AND_ISSUE_ADS
        &ad_active_status=ALL
        &fields=ad_creation_time%2Cad_creative_body2Cpage_name%2Cdemographic_distribution
        &limit=100
        &ad_reached_countries=NL
        &search_terms=.

There are of course a number of clients that can perform API calls, in the following pilot data analysis we are using a Python implementation

Interface example:

3. Auditing Facebok Ads¶

Auditing Facebok Ads - Data Collection¶

Max Woolf's facebook-ad-library-scraper it's the best out-the-box solution (as of mid 2021) to to retrive ads data from a Python client, since it requires minimal dependencies.

Usage¶

• Configure the script via config.yaml. Go to https://developers.facebook.com/tools/explorer/ to get a User Access Token, and fill it in with your token (it normally expires after a few hours but you can extend it to a 2-month token via the Access Token Debugger). Change other parameters as necessary.
• Generate a token clicking the button "Generate Access Token" and copy it, then paste t in the TOKEN.txt file in the root of this folder
• Install the requirements by running the following: !pip3 install requests tqdm plotly
• Run the scraper script:

!python fb_ad_lib_scraper.py

• Outputs: This script outputs three CSV files in an ideal format to be analyzed.

fb_ads.csv: The raw ads and their metadata.
fb_ads_demos.csv: The unnested demographic distributions of people reached by ads, which can be mapped to fb_ads.csv via the ad_id field.
fb_ads_regions.csv: The unnested region distributions of people reached by ads, which can be mapped to fb_ads.csv via the ad_id field.

The following notebook automatically extracts over 500 ads by querying keywords related to housing credit: 'hypotheek', 'hypotheekadvies', 'huizen te koop', 'huis kopen' filtering therefore the ones related to the housing credit and setting a manageable limit for the API.

• The data: The fields include all details about the ads, like creation time, link, description, and caption. Moreover, each ad is associated with a funding entity and a source page, this source page is normally the marketer or the original Facebook page associated with the ad, finally, there is information on the amount spend, and demographics like gender and regions.

    Age groups:  
      - 18-24
      - 25-34
      - 35-44
      - 45-54
      - 55-64
      - 65+
  
    Gender groups:  
      - male
      - female
      - unknown
  
    Regions:  
      - Noord-Holland
      - Zuid-Holland
      - Gelderland
      - North Brabant
      - Utrecht
      - Groningen
      - Overijssel
      - Flevoland
      - Limburg
      - Friesland
      - Zeeland

Extracting and reading the data¶

Auditing Facebok Ads - Descriptive Statistics¶

The following section is focused on the statistical methodologies ffor auditing Facebook Ads data, it is important to note that no hypothesis testing is performed in the current pilot data analysis, meaning that no correlation analysis or causal effects are studied. The purpose of this section is to understand the key insights of the data and therefore ask questions about ads practices.

Ads Impressions Period¶

The ads run following the configuration set up that was made by the campaign creator, normally the conditions are that the campaign will run until the funds are exhausted, and there is a declared min and max per ad. Following this logic, some ads can run for hours, but some others for days, here we find the average, max, min and count of each ad

The period of the dataset that was extracted from the API is 2019-04, to 2021-09. It is taken this period of time, since one can't specify the period in the API call, it seems this is all the information available.

The campaigns are online in average 8 days and 15 hours, with a standard deviation of 15 days, the max campaign duration is 69 days.

The longest campaign was the following ad from the ASN Bank. This ad was online for 69 days.

Here the actual URL to the archive:
https://www.facebook.com/ads/library/?active_status=all&ad_type=political_and_issue_ads&country=NL&q=752867881939190&sort_data[direction]=desc&sort_data[mode]=relevancy_monthly_grouped&search_type=keyword_unordered&media_type=all

Checking authenticity of links to find potential scammers¶

As we could see, the ads would normally link you to the seller page. However, some mischievous advertisers and marketers sometimes would use the ads to promote certain links or websites, this website would link to a scam page or sometimes viruses. One way to check the authenticity of the links is to actually compare the link URL to the name of the original page. Following this approach if the page name is ASN Bank then the associated link is asnbank.nl or similar. Obviously, there would be a number of false positives, nevertheless is a good indication of dubious links.

It is used the Levenshtein distance to measure the similarity between the link and the page name, the closer to 0 the similar the names.

As we can see there are a lot of ads of politics, and all the websites are legit, this is one of the limitations of the Facebook Ads Library

The following example shows how a link is similar to the page name and therefore is having a distance closer to 0.

Checking the number of ads by advertiser to find odd patterns¶

Who is actually the website that is putting more ads on Facebook?
To answer this, we simply make a count of unique ads by each one of the pages, normally, certain campaigns of the same party run in parallel.

Only Nationaal hypotheek plan is a website that is directly related to credit housing

Auditing the Demographic Groups Distribution¶

The Facebook Ads Library does not include any information of the actual people that were exposed to the ads, this would actually mean a violation of people's personal information and therefore a legal problem for Facebook. Instead, the API provides aggregated statistics of the demographics groups that were displayed by each ad. For example, the ad number 752867881939190 was displayed with the following distribution.

We could therefore ask different questions such as. Which groups are longer exposed?

Distributions by Group¶

Moreover, we could explore which groups tend to be more microtargeted. Meaning that a campaign creator would only focus on a particular combination of demographical attributes. For instance "Female 18-24 group". The highest the percentage the more microtargeted.

From the ads we have we can see that male between 35 and 54 are more microtargeted than female, and it happens the opposite with young woman and woman 65+

Important to note that the group 18-24 is the age group were the ads are shown quicker and more microtargeted. (the blue color)

No matter the age group, normally the campaigns would run proportionally, the youngest group 13-17 still received some ads. The explanation for those cases is that the advertiser did not specify the age range to show the ad

Microtargeted Ads¶

Given the above observations, we could explore further the more clever marketing tricks, this microtargeting that can reach audiences in the most optimal way, like for instance targeting one specific demographic group. To gather those we get only the ads are only displayed by region, gender and age quickly exhausting the budget (the shortest amount of time).

Top 3 ads that were specifically targeted to demographic groups in a quick period of time exhausting the budget reaching maximum audience.

Here the actual URLs to the archive:

• 18-24 male https://www.facebook.com/ads/library/?active_status=all&ad_type=political_and_issue_ads&country=NL&q=3327051823976757&sort_data[direction]=desc&sort_data[mode]=relevancy_monthly_grouped&search_type=keyword_unordered&media_type=all

• 65+ female https://www.facebook.com/ads/library/?active_status=all&ad_type=political_and_issue_ads&country=NL&q=845655732843719&sort_data[direction]=desc&sort_data[mode]=relevancy_monthly_grouped&search_type=keyword_unordered&media_type=all

• 18-24 female https://www.facebook.com/ads/library/?active_status=all&ad_type=political_and_issue_ads&country=NL&q=531953940763325&sort_data[direction]=desc&sort_data[mode]=relevancy_monthly_grouped&search_type=keyword_unordered&media_type=all

Top ad by Region¶

Similarly to the demographic groups, one can aggregate the ads impressions by Dutch province.

The reading is that 4 out of 100 Ads shown in Noord-Brabant aren't shown anywhere else. On the other hand, 3 out of 3 ads that were shown in Paramaribo were specifically designed for that region, these correspond to 1 campaign.

Ads Topics¶

Performing N-grams analysis on the text of the ads. It is considered 1-grams to 4-grams terms using Dutch and English dictionaries, finally terms frequency and inverse terms frequency are compared.

The n-grams analysis does not necessarily present key insights, mostly we can see generic terms, what would be interesting to analyse is the consistency of the advertisement and the actual campaign plans. Not only that, but this workflows can be adapted to be iterable so that we could see what topics are shown by demographic groups and regions.

Related work and Conclusion¶

There is also some interesting work conducting similar analysis, for example, Sara Kingsley et al. paper Auditing Digital Platforms for Discrimination in Economic Opportunity Advertising, shows how we systematically collect ads and find discrimination based on the demographics or locations, for US data it was possible in 2020 to gather ads tagged as, Credit, Housing, and Employment.

Also, Muhammad Ali et al. paper Discrimination through optimization: How Facebook’s ad delivery can lead to skewed outcomes explore the concept of skewness in the ad delivery process, where the background of the advertiser when designing parameter for campaigns can lead to discrimination when is only focused on optimizing impressions and budget.

In this notebook, we have focused on the Dutch credit housing finding not surprising insights, however, what is more, important is to open the discussion whether Facebook should be forced to open their Ads Library to access all types of ads and not only the "Social Issues, Elections or Politics" category.

Date: Sep 2021
Author: Pedro V Hernández Serrano
License: Attribution 4.0 International (CC BY 4.0)