Commit Graph

45 Commits

Author SHA1 Message Date
agres 7552d43702 Bigger batch size 2025-03-23 21:55:11 +01:00
agres e73752c015 Fixed counter issue where the accumilation would be calculated wrongly and the requests were not at the fullest capacity 2025-03-23 21:54:44 +01:00
agres 164ea9aca9 'fix' for a bug 2025-03-23 20:49:34 +01:00
agres af6b3dba13 The most convoluted, code-efficient, bloated, boilerplated, duplicated unneccecary undocumented, soon to be regretted code 2025-03-23 20:49:00 +01:00
Chris Kiriakou 625f09cfdf Change data-docker dir in code-block description 2025-03-23 19:38:01 +01:00
Chris Kiriakou 99a19d38d6 Fix stupid typo that costed dome 5 hours to debug. 2025-03-23 19:35:49 +01:00
Dominik ff9d726b47 Feat/import gdrp data (#26)
* Some simple code for extracting data from the jsons

* Jupiter Notebook

* Mac specific gitignore

* Fixed finding paths to floders

* Delete src/gdpr_data directory

* Updated gitignore to include my testing file

* Added the standard saving path for the database in the database handler, this way multiple files dont have to be updated when moving database position

* Moved the API usage wrappers into an own file, added a function for getting multiple track_ids at once, this still needs to be tested more

* Further code for extracting data from the gdpr files

* Forgor

* Final&Tested version of get_multiple_tracks_information endpoint

* Further functionality: The code now extracts the id of each listened song and makes a api call to get info about these songs via the multiple tracks api. Furthermore we track the songs witch the call is made for already and skip these

* Added function to map catalouged ids into the play history

* Added args parser to runtime program, cleaned up some code

* Fixed a bug where the database would always try to create tables, eaven if it exists

* Added some small text for clean interface

* Some final fixes to actual code, fixed db bug, reversed the order of database entries

* Some documentation

* Added -export args to docker runtime

* fix
2025-03-23 18:48:57 +01:00
Dominik 535225392f Merge pull request #24 from agresdominik/feat/containerize_app
Feat/containerize app
2025-03-22 10:06:02 +01:00
Chris Kiriakou 71e9db8738 Remove unused tag. 2025-03-21 19:17:12 +01:00
Chris Kiriakou fd1d8e6874 Change the directory location 2025-03-21 19:13:46 +01:00
Chris Kiriakou 0531a346b5 Add new folders 2025-03-21 19:12:44 +01:00
Chris Kiriakou aeb7c49068 Create improved Dockerfile with volume mounts for storing data
* Add volume mounts to Dockerfile to enable persisten storing of
  database and tokens when using docker
* Makefile helps with cleanup and building the Dockerfile
* Startup fires the predictify runtime inside the container
* `.dockerignore` contains files that are not needed by the docker,
  e.g. files that would otherwise slow build process down
2025-03-21 19:07:30 +01:00
Chris Kiriakou 3922c7640e Add documentation for docker and small visual improvements 2025-03-21 19:05:00 +01:00
Chris Kiriakou 38e0c69bea Move .env file to config dir, change db path for improved project structure
* Change path of db and token to `data` which helps to sperate source
  code from data files like `tokens.json` & `spotify_scraped.db`
* Change path of `.env` `src/env/` to `config/` for the same reason as
  mentioned above
* Added the newly created files to `.gitigore` accordingly
2025-03-21 18:57:13 +01:00
Chris Kiriakou 2d4e951693 Move Dockerfile & startup script to sperate directory 2025-03-21 18:54:07 +01:00
agres cf7ede5495 Fix startup 2025-03-19 21:05:33 +01:00
Dominik e6e342ea8e Merge pull request #22 from agresdominik/staging
Staging to Main
2025-03-19 20:55:56 +01:00
Dominik 29deb2c971 Merge pull request #21 from agresdominik/feat/containerize_app
Create a custom Dockerfile to decouple application dependencies from …
2025-03-19 20:55:03 +01:00
Chris Kiriakou 3fe19fd41f Fix Dockerfile. 2025-03-19 20:54:13 +01:00
Dominik 1ad803bbfb Merge pull request #19 from agresdominik/feat/database_handler
Feat/database handler
2025-03-19 19:55:03 +01:00
agres 601b94e405 Added timer to avoid working with cronjobs 2025-03-19 19:54:06 +01:00
agres 6b35a34826 refacor & simple logger 2025-03-19 19:38:22 +01:00
agres 7db5c51867 Big commit 2025-03-19 18:20:49 +01:00
Chris Kiriakou ae477c01a4 Merge branch 'staging' into feat/containerize_app 2025-03-19 17:55:35 +01:00
Dominik e574c6bc00 Feat/auth multiple tokens integration (#15)
* Implemented a multiple token read/write/refresh function for the authentificator

* Rework auth with simple authentificate
2025-03-19 17:44:17 +01:00
Chris Kiriakou 9b9002e751 Create a custom Dockerfile to decouple application dependencies from host.
* Create a dockerfile based on alpine.
* Create a start script to startup scraper.
2025-03-19 16:56:23 +01:00
Dominik 57ac860f1f Merge pull request #11 from agresdominik/feat/scraping_implementation
Feat/scraping implementation
2025-03-19 16:31:39 +01:00
agres 9235b5a844 Gitignore 2025-03-19 16:27:48 +01:00
agres 77654f13c9 Big commit 2025-03-19 16:27:25 +01:00
agres 55eede61cc Added pytest and config file 2025-03-19 01:36:50 +01:00
agres 8d407d3d52 Added pytest, excluded tests from precommit 2025-03-19 01:22:38 +01:00
agres 12096b2ec0 Fixed doc 2025-03-19 01:07:42 +01:00
agres c3e3db87cc Added test newline 2025-03-19 01:06:52 +01:00
agres bb3c37ca48 Pre commit hook 2025-03-19 01:05:51 +01:00
agres e6e7888613 Added tokens file to gitignore 2025-03-18 21:44:22 +01:00
agres eb6a44a9b6 Implemented a token refresher/rotator and a token file so the user does not have to auth each time the code is ran 2025-03-18 21:43:40 +01:00
agres 668f840fff Authors Added 2025-03-18 19:48:46 +01:00
agres 5091319f27 Readme extended with auth documentation 2025-03-18 19:45:52 +01:00
agres fb0911d893 A basic auth framework 2025-03-18 19:44:36 +01:00
agres 5b1be7bbe5 A Example env file 2025-03-18 19:43:56 +01:00
agres ee76aabe58 Typo 2025-03-18 17:28:24 +01:00
agres 34c601986c Added vscode folder to gitignore and fixed md 2025-03-18 17:27:58 +01:00
agres e6a1562155 Readme start 2025-03-18 17:26:50 +01:00
agres 1d6f07ef18 Initial setup for scraper 2025-03-18 17:16:18 +01:00
Dominik 00172dee0c Initial commit 2025-03-18 17:04:08 +01:00