Lerking/web-scraping-python
1
0
Fork 0
mirror of https://github.com/Lerking/web-scraping-python.git synced 2025-07-21 19:11:04 +02:00
Code Issues Packages Projects Releases Wiki Activity

1st example done. /JL

Browse Source
This commit is contained in:
Jan Lerking
2023-05-17 17:17:01 +02:00
parent 2bb9d31795
commit abc0a62dce
3 changed files with 36 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
README.md
View File

@@ -1,30 +1,30 @@
# Python repo template
[![GitHub forks](https://img.shields.io/github/forks/Lerking/python-repo-template.svg?style=social&label=Fork&maxAge=2592000)](https://GitHub.com/Lerking/python-repo-template/forks/)
[![GitHub stars](https://img.shields.io/github/stars/Lerking/python-repo-template.svg?style=social&label=Star&maxAge=2592000)](https://GitHub.com/Lerking/python-repo-template/stargazers/)
[![GitHub watchers](https://img.shields.io/github/watchers/Lerking/python-repo-template.svg?style=social&label=Watch&maxAge=2592000)](https://GitHub.com/Lerking/python-repo-template/watchers/)
# Python web scraping examples
[![GitHub forks](https://img.shields.io/github/forks/Lerking/web-scraping-python.svg?style=social&label=Fork&maxAge=2592000)](https://GitHub.com/Lerking/web-scraping-python/forks/)
[![GitHub stars](https://img.shields.io/github/stars/Lerking/web-scraping-python.svg?style=social&label=Star&maxAge=2592000)](https://GitHub.com/Lerking/web-scraping-python/stargazers/)
[![GitHub watchers](https://img.shields.io/github/watchers/Lerking/web-scraping-python.svg?style=social&label=Watch&maxAge=2592000)](https://GitHub.com/Lerking/web-scraping-python/watchers/)
[![forthebadge made-with-python](http://ForTheBadge.com/images/badges/made-with-python.svg)](https://www.python.org/)
[![GitHub version](https://badge.fury.io/gh/Lerking-python-repo-template.svg)](https://github.com/Lerking/python-repo-template)
[![GitHub version](https://badge.fury.io/gh/Lerking-web-scraping-python.svg)](https://github.com/Lerking/web-scraping-python)
[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Lerking/python-repo-template.github.io/graphs/commit-activity)
[![GitHub contributors](https://img.shields.io/github/contributors/Lerking/python-repo-template.svg)](https://GitHub.com/Lerking/python-repo-template/graphs/contributors/)
[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Lerking/web-scraping-python.github.io/graphs/commit-activity)
[![GitHub contributors](https://img.shields.io/github/contributors/Lerking/web-scraping-python.svg)](https://GitHub.com/Lerking/web-scraping-python/graphs/contributors/)
[![Github file count](https://img.shields.io/github/directory-file-count/Lerking/python-repo-template)]()
[![Github file count](https://img.shields.io/github/directory-file-count/Lerking/web-scraping-python)]()
[![GitHub issues](https://img.shields.io/github/issues/Lerking/python-repo-template.svg)](https://GitHub.com/Lerking/python-repo-template/issues/)
[![GitHub issues-closed](https://img.shields.io/github/issues-closed/Lerking/python-repo-template.svg)](https://GitHub.com/Lerking/python-repo-template/issues?q=is%3Aissue+is%3Aclosed)
[![Percentage of issues still open](http://isitmaintained.com/badge/open/Lerking/python-repo-template.svg)](http://isitmaintained.com/project/Lerking/python-repo-template "Percentage of issues still open")
[![Average time to resolve an issue](http://isitmaintained.com/badge/resolution/Lerking/python-repo-template.svg)](http://isitmaintained.com/project/Lerking/python-repo-template "Average time to resolve an issue")
[![GitHub issues](https://img.shields.io/github/issues/Lerking/web-scraping-python.svg)](https://GitHub.com/Lerking/web-scraping-python/issues/)
[![GitHub issues-closed](https://img.shields.io/github/issues-closed/Lerking/web-scraping-python.svg)](https://GitHub.com/Lerking/web-scraping-python/issues?q=is%3Aissue+is%3Aclosed)
[![Percentage of issues still open](http://isitmaintained.com/badge/open/Lerking/web-scraping-python.svg)](http://isitmaintained.com/project/Lerking/web-scraping-python "Percentage of issues still open")
[![Average time to resolve an issue](http://isitmaintained.com/badge/resolution/Lerking/web-scraping-python.svg)](http://isitmaintained.com/project/Lerking/web-scraping-python "Average time to resolve an issue")
[![GitHub pull-requests](https://img.shields.io/github/issues-pr/Lerking/python-repo-template.svg)](https://GitHub.com/Lerking/python-repo-template/pull)
[![GitHub total-pull-requests](https://badgen.net/github/prs/Lerking/python-repo-template)](https://GitHub.com/Lerking/python-repo-template/pull)
[![GitHub pull-requests](https://img.shields.io/github/issues-pr/Lerking/web-scraping-python.svg)](https://GitHub.com/Lerking/web-scraping-python/pull)
[![GitHub total-pull-requests](https://badgen.net/github/prs/Lerking/web-scraping-python)](https://GitHub.com/Lerking/web-scraping-python/pull)
Coding time for this reposotory.</br>
[![wakatime](https://wakatime.com/badge/user/d43f2852-fd6f-45b4-b713-558ad18204d4/project/dcf411f9-6d3e-4b14-9290-5ed419cf4012.svg)](https://wakatime.com/badge/user/d43f2852-fd6f-45b4-b713-558ad18204d4/project/dcf411f9-6d3e-4b14-9290-5ed419cf4012)
[![wakatime](https://wakatime.com/badge/user/d43f2852-fd6f-45b4-b713-558ad18204d4/project/6b4d8af5-fd7e-4935-8116-60b9b13b44e3.svg)](https://wakatime.com/badge/user/d43f2852-fd6f-45b4-b713-558ad18204d4/project/6b4d8af5-fd7e-4935-8116-60b9b13b44e3)
See the wiki for usage and examples.
[python-repo-template wiki page](https://github.com/Lerking/python-repo-template/wiki)
[web-scraping-python wiki page](https://github.com/Lerking/web-scraping-python/wiki)
Visitors since repo creation.</br>
![Visitor Badge](https://visitor-badge.laobi.icu/badge?page_id=Lerking.python-repo-template)
![Visitor Badge](https://visitor-badge.laobi.icu/badge?page_id=Lerking.web-scraping-python)

0
main.py
View File

19
mechanicalsoup_example.py Normal file
View File

@@ -0,0 +1,19 @@
import mechanicalsoup
def main():
browser = mechanicalsoup.StatefulBrowser()
browser.open("https://www.sas.dk/book/flights/?search=RT_CPH-NYC-20230521-20230528_a1c0i0y0&view=upsell&bookingFlow=points&sortBy=rec,rec&filterBy=all,all&out_class=BUSINESS&out_sub_class=SAS%20BUSINESS%20PRO&out_flight_number=SK1462,SK907")
#offers = browser.page.find('td', class_='slide-plus up-grid-prods flight-products')
#offers = browser.page.find_all('span', class_='fareprice-ubc')
offers = browser.page.find_all('link')
print(browser.get_url())
print(browser.get_current_page().title.text)
#print(browser.page)
print(offers)
if __name__ == "__main__":
main()
'''
<td flight-products="" tabindex="0" class="slide-plus up-grid-prods flight-products" aria-expanded="false"><div class="fareprice-parent fare-class-and-price-container"><span>Plus</span><!----><!----><!----><span aria-hidden="true" class="fareprice-ubc"><!----><p aria-hidden="true" class="discounted-point strike margin-auto mobilepoints max-digits">352 926 <span class="desktop-only">p</span></p></span><!----><!----><!----><!----><span role="presentation" class="sr-only"><!----><!----><span>PLUS Class fare: 352 926 <span class="points-units"> p</span><!----></span><!----><!----><!----><!----></span><!----><!----></div></td>
'''