42fac846aa
Merge branch 'master' of https://github.com/1kamma/supermarket-scraping
2022-10-05 03:52:43 +03:00
korenlazar
9b6f63a7f0
Added the chain Yeinot Bitan (also to tests).
...
Changed price with promos to include only regular promotions.
Added filtering of promotions including too many items.
2022-10-04 13:36:29 +03:00
korenlazar
ceff48dbd9
Fixed the bug with cerberus_web_client.py by working with Selenium. To login each chain working with it must have a username for login with Selenium. in this mechanism, a path to a gz file is returned instead of url
...
Added the option to output a prices json file in main.py under --prices-with-promos, where the prices are updated by the latest promotions (under the 'final_price' key, where 'price' represents the price before promotions).
Fixed small bug of BinaWebCleint by checking that filename does not contain 'null'.
Changed Hierarchy of chains such that it includes the webclients.
Added the date to the output filenames to start storing the data over time.
Black formatting (according to pip 8 guidelines).
Changed the chains_dict in main to a constant one.
2022-10-04 11:42:36 +03:00
KorenLazar
90cab0a2e1
Minor changes
2021-08-18 11:32:04 +03:00
KorenLazar
87b6fbe2b0
Changed ClubID enum class to include a string field used for printing, and define ClubID.OTHER as a default value for the class to handle invalid inputs.
2021-08-18 11:30:31 +03:00
KorenLazar
c1281cb312
Added a test for scraping the promotions and exporting them to xlsx files.
2021-08-16 23:09:10 +03:00
KorenLazar
1a88ed6e01
minor changes
2021-08-16 23:08:04 +03:00
KorenLazar
1a6707341d
Logical fixes in promotions scraping and calculation.
2021-08-16 23:07:07 +03:00
KorenLazar
844a106c57
Added tqdm
2021-08-16 23:05:16 +03:00
KorenLazar
13991aaa40
Documentation and minor changes
2021-08-16 14:05:22 +03:00
KorenLazar
b3d410306d
Removed filtering by PRODUCTS_TO_IGNORE
2021-08-16 14:04:46 +03:00
KorenLazar
58bb04f1dd
Added get_all_promos_tags function and included the non-full promotions file in the promotions collection.
2021-08-16 12:42:38 +03:00
KorenLazar
27b45a4999
finished implementing exporting promotion to xlsx table and automatically opening the xlsx file
2021-06-01 21:00:40 +03:00
3ae8d02836
correction, by comments and suggestions of Koren
2021-04-29 17:55:21 +03:00
e1f43772b9
now excel is working
2021-04-17 20:49:08 +03:00
d4ba19bf41
remove_unneeded
2021-04-17 12:07:08 +03:00
2a4b6562b7
change encoding
2021-04-17 12:06:25 +03:00
KorenLazar
9f5464317d
Has added tests for the promotion functions for Shufersal and CoOp. Also added minor design changes in promotion.py and item.py
2021-03-08 14:13:30 +02:00
KorenLazar
c86fc7c1ab
Moved to writing solely to CSV. Added some columns and drastly improved the logics behind price after promotion column.
2021-02-25 20:54:44 +02:00
KorenLazar
8aa33cbcda
added columns to csv: price after promotion, discount in percentage and promotion type (regular/club/credit card).
2021-02-23 08:27:00 +02:00
KorenLazar
18f3fa32b9
has added many chains
2021-02-06 21:42:31 +02:00
KorenLazar
5aa4cd734d
changed chains' members to be 'immutable static'
2021-02-06 15:57:05 +02:00
KorenLazar
3a57edf5af
has added RamiLevi to the chains collection
2021-02-06 14:41:04 +02:00
KorenLazar
d7e5b709f8
has added a CSV format promotions file when running --promos. Item.py was added for moduling.
2021-01-28 16:25:38 +02:00
KorenLazar
6bcbdee486
added sorting by promotiong length (shorter first)
2021-01-09 16:35:55 +02:00
KorenLazar
ee35e3436c
Has added Zol Vebengadol. Fixed small bug in Co-Op.
2020-12-26 17:18:21 +02:00
KorenLazar
be47a5ad81
has added Co-Op chain. Extendability to other chain has significantly improved as well.
2020-12-17 22:48:10 +02:00
KorenLazar
4c0eba1771
added --find_promos_by_name option
2020-12-16 16:09:43 +02:00
KorenLazar
db6bfb3632
changed create_bs mechanism and moved to simply printing to stdout in --find_store_id and --price
2020-12-15 11:28:17 +02:00
KorenLazar
975447dcd7
splitted to multiple files
2020-12-13 13:33:07 +02:00
