Views:
112β
Votes: 5β
Tags:
support
data-explorer
data-dump
Link:
π See Original Question on Meta Stack Exchange β§ π
URL:
https://meta.stackexchange.com/q/372300
Title:
Use cron to get SE Data Explorer data when it's updated
ID:
/2021/11/27/Use-cron-to-get-SE-Data-Explorer-data-when-it_s-updated
Created:
November 27, 2021
Edited: November 27, 2021
Upload:
September 15, 2024
Layout: post
TOC:
false
Navigation: false
Copy to clipboard: false
According to this answer from 2012, the SE Data Explorer data is updated every Sunday at 3am. If true, is it possible to schedule a cron job to run every Monday and execute this query?
When the query finishes it would need to download the results in CSV format somewhere.
More specifically - is there a simple way I can run a query against SEDE directly from the terminal?
Concerns
I imagine itβs a little complicated getting cron authenticated by Stack Exchange Data Explorer. Iβve never run cron with my own User ID. It always runs as sudo
either every 15 minutes to execute updatedb
or via scripts in /etc/cron.daily
, /etc/cron.weekly
and /etc/cron.monthly
. SE always wants authentication when I run a DE query as a βnormalβ user. I have never used cron for signing into a site, running a query, waiting for results, clicking a βdownloadβ button, and logging out. It sounds horribly complicated giving cron an X11 session, a GUI Desktop and a browser.