velocitatem/cvfs
1
0
Fork 0
mirror of https://github.com/velocitatem/cvfs.git synced 2026-05-31 08:43:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Daniel Alves Rösel
2026-04-02 18:47:14 +02:00
committed by GitHub
commit 90ad5e0260
94 changed files with 7797 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
ml/data/README.md Normal file
View File

@@ -0,0 +1,5 @@
# Data
Some thoughts on processing data: In a lot of cases you will get data not in an s3 bucket or anything glamarous and for doing anything in terms of modelling you need the data locally but then when you maybe have a 1TB dataset you want 10GB locally and then you upload to a GPU rich server and there you will want all of teh data. How can you managed this data well? What are best practices?
Huggingface lets you upload up to 300 gigs of data into a dataset.