saret/supermarket-scraping
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Commit Graph

  • 6f527b12d5 prefs master saret 2022-10-05 12:42:53 +03:00
  • b8ccd6dcf7 fix of download directory saret 2022-10-05 12:38:52 +03:00
  • 6755ff5caf Merge pull request #3 from 1kamma/master Koren Lazar 2022-10-05 07:51:58 +03:00
  • 42fac846aa Merge branch 'master' of https://github.com/1kamma/supermarket-scraping saret 2022-10-05 03:52:43 +03:00
  • d047ffdcc2 added options for headless computers, changed the downloa path to raw_files saret 2022-10-05 03:37:43 +03:00
  • 9b6f63a7f0 Added the chain Yeinot Bitan (also to tests). korenlazar 2022-10-04 13:36:29 +03:00
  • 86ff2ca7b7 Fixed small bug in valid_store_id_by_chain function korenlazar 2022-10-04 12:11:44 +03:00
  • b1737839ce Fixed bug with Shufersal Scraping by changing xml files category back to normal Enum. korenlazar 2022-10-04 12:09:42 +03:00
  • 7b63eab7bd leftover from last commit korenlazar 2022-10-04 11:42:57 +03:00
  • ceff48dbd9 Fixed the bug with cerberus_web_client.py by working with Selenium. To login each chain working with it must have a username for login with Selenium. in this mechanism, a path to a gz file is returned instead of url korenlazar 2022-10-04 11:42:36 +03:00
  • b5db721a3d Merge pull request #6 from korenLazar/test-scraping korenLazar 2021-08-18 12:26:23 +03:00
  • 90cab0a2e1 Minor changes KorenLazar 2021-08-18 11:32:04 +03:00
  • 87b6fbe2b0 Changed ClubID enum class to include a string field used for printing, and define ClubID.OTHER as a default value for the class to handle invalid inputs. KorenLazar 2021-08-18 11:30:31 +03:00
  • 322995ba15 Added TODO for ordering the argparse KorenLazar 2021-08-18 11:16:25 +03:00
  • 294dee8cc2 Added test for searching different files' urls. Specifically, asserting the searching non-full files does not yield urls of full files. KorenLazar 2021-08-17 13:08:39 +03:00
  • cffdd84086 Added specific searching for the download url of non-full promotions and prices files. Changed return value of get_download_url accordingly. KorenLazar 2021-08-17 13:06:42 +03:00
  • 3770352d04 Added new requirements to requirements.txt KorenLazar 2021-08-17 09:35:20 +03:00
  • 63fec1490c Added new requirements to requirements.txt KorenLazar 2021-08-17 09:18:45 +03:00
  • c1281cb312 Added a test for scraping the promotions and exporting them to xlsx files. KorenLazar 2021-08-16 23:09:10 +03:00
  • 1a88ed6e01 minor changes KorenLazar 2021-08-16 23:08:04 +03:00
  • 9b0ab013c9 Added requirements to requirements.txt KorenLazar 2021-08-16 23:07:32 +03:00
  • 1a6707341d Logical fixes in promotions scraping and calculation. KorenLazar 2021-08-16 23:07:07 +03:00
  • 844a106c57 Added tqdm KorenLazar 2021-08-16 23:05:16 +03:00
  • c793057623 Documentation and minor changes KorenLazar 2021-08-16 14:06:54 +03:00
  • 13991aaa40 Documentation and minor changes KorenLazar 2021-08-16 14:05:22 +03:00
  • b3d410306d Removed filtering by PRODUCTS_TO_IGNORE KorenLazar 2021-08-16 14:04:46 +03:00
  • 62089dd538 Merge pull request #5 from korenLazar/export-promotions-to-xlsx-table korenLazar 2021-08-16 12:51:48 +03:00
  • 03ff6d5281 Changed create_items_dict function to included non-full prices file in the items dictionary. Changed log_products_prices to work with an items dictionary and a __repr__ function of the Item class. KorenLazar 2021-08-16 12:44:32 +03:00
  • e09b2da4a1 removed get_all_deals function KorenLazar 2021-08-16 12:43:01 +03:00
  • 58bb04f1dd Added get_all_promos_tags function and included the non-full promotions file in the promotions collection. KorenLazar 2021-08-16 12:42:38 +03:00
  • ebb1e912b9 Change INFO logging format KorenLazar 2021-08-16 12:40:06 +03:00
  • 98dcc1c33d Add price_by_measure member to Item object KorenLazar 2021-08-16 12:39:28 +03:00
  • 8a726ff605 Merge pull request #4 from korenLazar/export-promotions-to-xlsx-table korenLazar 2021-06-17 10:36:20 +03:00
  • 27b45a4999 finished implementing exporting promotion to xlsx table and automatically opening the xlsx file KorenLazar 2021-06-01 21:00:40 +03:00
  • ec505dba67 minor rephrasing in documentation KorenLazar 2021-05-18 14:34:11 +03:00
  • 3ae8d02836 correction, by comments and suggestions of Koren saret 2021-04-29 17:55:21 +03:00
  • 5caf3e495c mistake in the requierments fixed 0.1 saret 2021-04-17 22:36:18 +03:00
  • e1f43772b9 now excel is working saret 2021-04-17 20:49:08 +03:00
  • e740b122ff Merge pull request #1 from 1kamma/master korenLazar 2021-04-17 18:34:11 +03:00
  • d4ba19bf41 remove_unneeded saret 2021-04-17 12:07:08 +03:00
  • 2a4b6562b7 change encoding saret 2021-04-17 12:06:25 +03:00
  • 9f5464317d Has added tests for the promotion functions for Shufersal and CoOp. Also added minor design changes in promotion.py and item.py KorenLazar 2021-03-08 14:13:30 +02:00
  • c86fc7c1ab Moved to writing solely to CSV. Added some columns and drastly improved the logics behind price after promotion column. KorenLazar 2021-02-25 20:54:44 +02:00
  • 8aa33cbcda added columns to csv: price after promotion, discount in percentage and promotion type (regular/club/credit card). KorenLazar 2021-02-23 08:27:00 +02:00
  • 850d3963fe has added binaproject clients KorenLazar 2021-02-07 10:46:54 +02:00
  • 9983d07c2b replaced the member '_class_name' by the 'class.__name__' KorenLazar 2021-02-07 08:18:22 +02:00
  • 67bff9fa76 minor changes KorenLazar 2021-02-06 22:53:18 +02:00
  • 18f3fa32b9 has added many chains KorenLazar 2021-02-06 21:42:31 +02:00
  • 5aa4cd734d changed chains' members to be 'immutable static' KorenLazar 2021-02-06 15:57:05 +02:00
  • 3a57edf5af has added RamiLevi to the chains collection KorenLazar 2021-02-06 14:41:04 +02:00
  • d7e5b709f8 has added a CSV format promotions file when running --promos. Item.py was added for moduling. KorenLazar 2021-01-28 16:25:38 +02:00
  • 47c0d04ce4 Updated README.md with latest changes and directory names to be more meaningful KorenLazar 2021-01-28 14:13:34 +02:00
  • 6bcbdee486 added sorting by promotiong length (shorter first) KorenLazar 2021-01-09 16:35:55 +02:00
  • ee35e3436c Has added Zol Vebengadol. Fixed small bug in Co-Op. KorenLazar 2020-12-26 17:18:21 +02:00
  • be47a5ad81 has added Co-Op chain. Extendability to other chain has significantly improved as well. KorenLazar 2020-12-17 22:48:10 +02:00
  • 4c0eba1771 added --find_promos_by_name option KorenLazar 2020-12-16 16:09:43 +02:00
  • db6bfb3632 changed create_bs mechanism and moved to simply printing to stdout in --find_store_id and --price KorenLazar 2020-12-15 11:28:17 +02:00
  • 975447dcd7 splitted to multiple files KorenLazar 2020-12-13 13:33:07 +02:00
  • c16f0bada9 Merge branch 'master' of https://github.com/korenLazar/shufersal-scraping KorenLazar 2020-12-07 13:50:03 +02:00
  • 795d7050a5 has added gitignore file KorenLazar 2020-12-07 13:49:05 +02:00
  • dfca61c793 has fixed has_expired validation, added start_date and has_started validation, changed promotions sorting to max(update_date,start_date), KorenLazar 2020-12-07 13:48:16 +02:00
  • 8bfe4f1cb1 Update README.md korenLazar 2020-12-02 09:57:37 +02:00
  • 80787c90f1 Update README.md korenLazar 2020-12-02 09:55:47 +02:00
  • 50dd160a06 changed README extension to md KorenLazar 2020-12-02 07:00:22 +02:00
  • 6daf91cc08 has added requirements.txt file KorenLazar 2020-12-02 00:14:32 +02:00
  • 0bbaaa2ead has added README file KorenLazar 2020-12-02 00:04:52 +02:00
  • 26c1103571 Efficiency improvements were added KorenLazar 2020-11-29 10:09:51 +02:00
  • 18ea20f6fd First commit KorenLazar 2020-11-24 15:13:44 +02:00