Sebenarnya, jelas bukan harus. Maksudnya, kamu tentunya bebas memilih jalur karir remote work-mu yang manapun. Namun, pengamatan kami di Remote Worker Indonesia menunjukkan suatu trend, bahwa pemula sekalipun, jika sudah menguasai tiga hal ini:
Python Fundamental
Mastering Git
Data Scraping
Akan cepat sekali bisa menembus remote work via Upwork.
"Wah semudah itu???"
Maaf-maaf, ga semudah itu juga.
Kamu tetap harus berproses menuntaskan banyak studi kasus web scraping. Nah, di course ini kamu akan menemukan banyak sekali studi kasus data scraping berikut ini:
Scraping situs indeed
Scraping situs Steam
Scraping situs carousel
Scraping detik
Apa itu Web Scraping?
Web Scraping adalah kegiatan untuk mengambil data dari suatu website dengan memanfaatkan tag-tag, class dan id atribut HTML.
Apa yang diperlukan untuk Web Scraping Pemula?
Untuk melakukan web scraping hal minimal yang harus dipahami adalah
Komponen Halaman Web
Mengetahui HTML Dasar
Python Dasar
Mengetahui Jaringan Dasar [misalnya HTTP Request]
Komponen Halaman Web
Komponen Halaman Web secara dasar terdiri atas 3 komponen utama seperti
HTML adalah komponen paling dasar yang berfungsi sebagai kerangka utama dalam pembuatan Web CSS digunakan agar web terlihat lebih indah seperti komposisi warna ukuran serta posisi, semua diatur menggunakan CSS ini JavaScript bisa digunakan sebagai backend ataupun frontend sebuah website supaya lebih interaktif dan mudah digunakan pengguna
Web Scraping StarterPack
Apa yang dibutuhkan untuk melakukan web scraping?, ada beberapa modul yang biasa digunakan untuk melakukan web scraping seperti
Instagram is currently one of the most important social networks around the world, specially in the western countries. With over 1 billion of monthly active users and 500 daily active users Instagram becomes a great opportunities for brands to connect with potential customers, improve their brand awareness and visibility and build customer loyalty. On the other hand, Instagram can also be a great chance to create interesting allies in the form of brand ambassadors, collaboration with influencers and business partners or to generate sales opportunities.
In this post I am going to show you how to take maximum advantage of Instagram to boost your business performance by automating most of the most time-consuming and burdensome tasks such as analyzing the most successful publications from competitors, tracking competitors stories or scraping Instagram profiles to extract some data like number of followers, posts or email addresses if found in the biography to find the right person to give visibility to your brand, partner with you or create sales opportunities.
However, if you would like to scrape from Instagram a very big number of profiles and posts, it might be better to use an API such as Apify, which uses proxies to not get banned after scraping a number of pages. You can have a look at its Instagram Scraper solution over here.
[email protected][a-z0-9\.\-+_]+\.com", soup, re.I] listinformation.append[[iteration,biography,externalurl,followers,following,businessacount,category,emails]] driver.quit[]Disclaimer: in case you are going to use these emails addresses to contact users or as an audience for Google Ads or Paid Social campaigns, you need to be careful as in the European Union there is a special legislation called GDPR which takes care of how personal information is collected and processed.
3.3.- Analyzing competitors’ best performant posts
Learning from the competitors and/or success stories can be a good exercise as you will discover which are the most engaging creatives or formats and you can orient your strategy accordingly. As a reminder, first of all the command that needs to be ran with instagram-scraper which returns the JSON data from the posts that are published from the Instagram account of your interest is:
instagram-scraper #Scraping is done anonymously. instagram-scraper -u -p #You log into your account and you make the scraping from there. Can be useful for private accounts.4.
In the example below, we will iterate through the JSON file, get the URL name [where the filename is included], the number of likes and the number of comments and we will store everything in an Excel file [as shown in the screenshot below] where the analysis can be done easier.
import json
import pandas as pd
with open['psg.json'] as json_file:
data = json.load[json_file] #We load the Json file
listposts = []
for x in range [len[data["GraphImages"]]]: #We iterate through the Json file and we get the variables through the keys
try:
media = data["GraphImages"][x]["display_url"]
likes = data["GraphImages"][x]["edge_media_preview_like"]["count"]
comments = data["GraphImages"][x]["edge_media_to_comment"]["count"]
listposts.append[[media,likes,comments]]
except:
continue
df = pd.DataFrame[listposts, columns=["Media","Likes","Comments"]]
df.to_csv['TestJason.csv', index=False] #We store the list in an Excel file by using Pandas
FAQs section
Which libraries do you need?
You will need os, json, beautifulsoup, cloudscraper, selenium, pandas and re.
What will you learn in this post?
You will learn how to scrape posts from Instagram with Python to find new affiliation opportunities, influencers or parters and track and analyze the competitors.
How long will it take?
The code is already created so the time would depend on the number of accounts you would like to scrape.