Menu

Post image 1
Post image 2
1 / 2
0

Automate Reddit DevOps Job Tracking with Python Scraper

DEV Community·IntelliTools·21 days ago
#UP94oaQl
Reading 0:00
15s threshold

import requests from bs4 import BeautifulSoup import csv # Reddit API endpoint for DevOps-related posts url = ' https://www.reddit.com/r/devops/search.json?q=jobs&sort=top&t=day ' headers = { ' User-Agent ' : ' Mozilla/5.0 ' } # Fetch and parse the page response = requests . get ( url , headers = headers ) data = response . json () # Extract post titles and links posts = data [ ' data ' ][ ' children ' ] job_data = [] for post in posts : title = post [ ' data ' ][ ' title ' ] link = post [ ' data ' ][ ' url ' ] location = post [ ' data ' ]. get ( ' subreddit ' , ' unknown ' ) job_data . append ([ title , link , location ]) # Export to CSV with open ( ' devops_jobs.csv ' , ' w ' , newline = '' , encoding = ' utf-8 ' ) as file : writer = csv . writer ( file ) writer . writerow ([ ' Title ' , ' Link ' , ' Location ' ]) writer .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More