The Cookie Machine - Click here to drag window

DUMMY TEXT - Real text set in assets/js/theCookieMachine.js

If you can read me, I'm broken!

Views: 112β€…    Votes:  5β€…
Tags: support   data-explorer   data-dump  
Link: πŸ” See Original Question on Meta Stack Exchange ⧉ πŸ”—

URL: https://meta.stackexchange.com/q/372300
Title: Use cron to get SE Data Explorer data when it's updated
ID: /2021/11/27/Use-cron-to-get-SE-Data-Explorer-data-when-it_s-updated
Created: November 27, 2021    Edited:  November 27, 2021
Upload: September 15, 2024    Layout:  post
TOC: false    Navigation:  false    Copy to clipboard:  false


According to this answer from 2012, the SE Data Explorer data is updated every Sunday at 3am. If true, is it possible to schedule a cron job to run every Monday and execute this query?

When the query finishes it would need to download the results in CSV format somewhere.

More specifically - is there a simple way I can run a query against SEDE directly from the terminal?


Concerns

I imagine it’s a little complicated getting cron authenticated by Stack Exchange Data Explorer. I’ve never run cron with my own User ID. It always runs as sudo either every 15 minutes to execute updatedb or via scripts in /etc/cron.daily, /etc/cron.weekly and /etc/cron.monthly. SE always wants authentication when I run a DE query as a β€œnormal” user. I have never used cron for signing into a site, running a query, waiting for results, clicking a β€œdownload” button, and logging out. It sounds horribly complicated giving cron an X11 session, a GUI Desktop and a browser.

⇧ How to find the saved Wifi Password in Ubuntu Ubuntu 20.04 on WSL2: Folder owner & group are being reset after each start  β‡©