Dylan De Faoite

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-11 19:44:40 +00:00

0658713f42 chore: remove unused dataset creation script

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-11 19:41:37 +00:00

b2ae1a9f70 feat(frontend): add page for scraping endpoint

eff416c34e fix(connectors): hardcoded source name in Youtube connector

Compare 2 commits »

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 23:28:33 +00:00

524c9c50a0 fix(api): incorrect dataset status update message

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 23:15:36 +00:00

2ab74d922a feat(api): support per-source search, category and limit configuration

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 22:48:08 +00:00

d520e2af98 fix(auth): missing email and username business rules

8fe84a30f6 fix: data leak when opening topics file

Compare 2 commits »

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 22:17:04 +00:00

dc330b87b9 fix(celery): process dataset directly in fetch task

7ccc934f71 build: change celery to debug mode

Compare 2 commits »

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 19:23:51 +00:00

a3dbe04a57 fix(frontend): option to delete dataset not shown after fail

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 19:17:45 +00:00

a65c4a461c fix(api): flask delegates dataset fetch to celery

15704a0782 chore(db): update db schema to include "fetching" status

6ec47256d0 feat(api): add database scraping endpoints

2572664e26 chore(utils): add env getter that fails if env not found

Compare 4 commits »

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 18:41:09 +00:00

17bd4702b2 fix(connectors): connector detectors returning name of ID alongside connector obj

53cb5c2ea5 feat(topics): add generalised topic list

0866dda8b3 chore: add util to always split evenly

Compare 3 commits »

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 18:18:46 +00:00

5ccb2e73cd fix(connectors): incorrect registry location

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 18:11:37 +00:00

2a8d7c7972 refactor(connectors): Youtube & Reddit connectors implement BaseConnector

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-10 18:08:03 +00:00

e7a8c17be4 chore(connectors): add base connector inheritance

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-09 21:29:11 +00:00

cc799f7368 feat(connectors): add base connector and registry for detection

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-09 20:55:15 +00:00

262a70dbf3 refactor(api): rename /upload endpoint

dylan created pull request dylan/crosspost#9

2026-03-09 20:53:41 +00:00

WIP: Automatic Scraping of dataset options

dylan created branch feat/automatic-scraping-datasets in dylan/crosspost

2026-03-09 20:53:16 +00:00

dylan pushed to feat/automatic-scraping-datasets at dylan/crosspost

2026-03-09 20:53:16 +00:00

ca444e9cb0 refactor: move connectors to backend dir

dylan pushed to main at dylan/birch-dots

2026-03-09 15:45:54 +00:00

fab1c53cb3 fix(bootstrap): entire file skip upon one package fail

dylan pushed to main at dylan/birch-dots

2026-03-09 15:40:41 +00:00

f29cf5e920 fix(packages): update incorrect docker compose package

dylan pushed to main at dylan/birch-dots

2026-03-09 15:36:53 +00:00

3ef8420a21 feat(packages): add rsync