Web Analytics

Backup Universal Analytics Data with Python

Vimal Joseph | | 4 min read

Universal Analytics End Of Life

Universal Analytics (UA) stopped processing data on July 1st, 2023. Google has announced a complete shutdown of the UA interface and APIs on July 1st, 2024. If you have already transitioned to Google Analytics 4 for your website data collection, this script can assist you in downloading data from your Universal Analytics properties. Downloading data from UA is not as straightforward as you might think. Google imposes a limit of 50,000 API calls per day, and if you have years of data to download with numerous dimensions and metrics, the script can quickly become complex.

Google Universal Analytics Data Archiving

I have created this Python script as an example to any one who is looking to create a script to fetches data using the Google Analytics Reporting API v4. It is designed to retrieve specific metrics and dimensions within a certain date range for a particular view (profile). This script is suitable for simple use cases, such as when you only need data over a short timeline or with a small number of dimensions or metrics.

 

 

Should You Backup Your GA3 Reports?

Universal Analytics Data: Should You Backup or Let Go? - This article will help you to decide whether you should backup GA3 (Universal Analytics) reports or not.

Free Tools to Backup GA3 Reports

Checkout this list of 3 tools that help you automate the backup of UA (GA3) reports. You can use these free tools to download the reports easily. One of the suggested tool is this Python script to backup GA3 (Universal Analytics) reports.

Download GA3 (Universal Analytics) Reports using Analytics Reporting API

We have made this script under GNU GPL V2 License and it is 100% Free Software. Here are the documentation to use this tool.

 

 

Requirements for the Python Script

  • Python 3
  • googleapiclient
  • oauth2client
  • pyyaml
  • argparse

Setup and Installation

  1. Clone the Repository:
git clone https://github.com/zyxware/ua-archive.git 
cd ua-archive
  1. Install Dependencies: It's recommended to use a virtual environment:
python3 -m venv venv source venv/bin/activate 
pip3 install google-api-python-client oauth2client argparse pyyaml
  1. Service Account and API Key:
  • Set up a service account in Google Cloud Console. - Follow Google Documentation for the detailed instructions.
  • Download the JSON key file.
  • Rename the file to ua-archive.json and place it in the project directory.
  1. Configure the Script:
    • Copy 'settings.yml.defualt' to 'settings.yml' and update 'api_key' with the name of json file generated and 'view_id' with your UA property's view ID.
    • Edit the reports_config.yml to add the reports you want to generate.

Usage

Run the script with Python:

python3 analytics_reporter.py --report_id 1 --start 2023-01-01 --end 2023-01-31

After successful execution, a CSV file named with the corresponding report name will be generated in the project directory.

Default Reports

The system supports the reports defined in reports_config.yml - You can modify the reports_conf.yml to add the dimensions and matrices you want to include in the report you want to generate. The full set of available dimensions and metrics are available on Google UA Dimensions & Metrics Explorer

reports:
  - id: 1
    name: "User Engagement Over Time"
    dimensions: ["ga:clientId", "ga:date"]
    metrics: ["ga:sessions", "ga:sessionDuration", "ga:pageviews"]
  - id: 2
    name: "User Journey Analysis"
    dimensions: ["ga:landingPagePath", "ga:exitPagePath", "ga:pagePath", "ga:previousPagePath"]
    metrics: ["ga:sessions", "ga:pageviews", "ga:entrances", "ga:exits"]
  - id: 3
    name: "Acquisition Channels and Campaign Performance"
    dimensions: ["ga:source", "ga:medium", "ga:campaign"]
    metrics: ["ga:sessions", "ga:users"]
  - id: 4
    name: "User Segmentation"
    dimensions: ["ga:userAgeBracket", "ga:userGender", "ga:userType"]
    metrics: ["ga:users", "ga:sessions", "ga:sessionDuration", "ga:pageviews"]
  - id: 5
    name: "Conversion Analysis"
    dimensions: ["ga:goalCompletionLocation", "ga:goalPreviousStep1", "ga:goalPreviousStep2", "ga:goalPreviousStep3"]
    metrics: ["ga:goal1Completions", "ga:goal1ConversionRate"]
  - id: 6
    name: "User Interaction with Content"
    dimensions: ["ga:pagePath", "ga:title"]
    metrics: ["ga:pageviews", "ga:uniquePageviews", "ga:avgTimeOnPage", "ga:entrances"]
  - id: 7
    name: "External Keyword and SEO Analysis"
    dimensions: ["ga:sourceMedium", "ga:landingPagePath"]
    metrics: ["ga:sessions", "ga:users", "ga:newUsers", "ga:pageviews", "ga:bounceRate", "ga:sessionDuration"]
    
  - id: 8
    name: "Internal Site Search Analysis"
    dimensions: ["ga:searchKeyword", "ga:searchStartPage", "ga:searchCategory"] 
    metrics: ["ga:sessions", "ga:users", "ga:pageviews", "ga:searchResultViews", "ga:searchUniques"]
  - id: 9
    name: "Device and Technology Analysis"
    dimensions: ["ga:deviceCategory", "ga:browser", "ga:operatingSystem"]
    metrics: ["ga:sessions", "ga:users", "ga:pageviews"]
  - id: 10
    name: "Custom User Events Tracking"
    dimensions: ["ga:eventCategory", "ga:eventAction", "ga:eventLabel"]
    metrics: ["ga:totalEvents", "ga:uniqueEvents"]

Do let us know if you need professional support in backing up your UA data. We can help you archive your Universal Analytics reports.

Sample Report: Channel Grouping

UA Channel Grouping

Channel Grouping is one of the most popular reports our clients are interested in downloading. To illustrate how the Channel Grouping report will appear after being backed up, we have created a demo report in Looker Studio. This demo provides a clear view of what to expect from the backup process.

To create a channel grouping report you can use the following configuration:

reports:
  - id: 1
    name: "Channel Grouping"
    dimensions: ["ga:date", "ga:channelGrouping"]
    metrics: ["ga:users", "ga:newUsers", "ga:sessions", "ga:bounceRate", ga:pageviewsPerSession, ga:avgSessionDuration]

Sample Report: Annotations in UA

UA Annotations

Annotations are another feature from Universal Analytics (UA) that many users want to download for reference. This feature is extensively used to mark major marketing events or changes happening on their websites. To assist in visualizing how annotations will be preserved, we have prepared a demo in Looker Studio. This demo showcases how the annotations will appear after being backed up.

Book a Consultation with our Google Analytics Expert

 

 

Related Topics