-
Notifications
You must be signed in to change notification settings - Fork 69
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
refactor: Added a generic
FileStream (still in active development!) (…
…#2654) * Initial impl * Implement a simple local file and directory * Fix types * Fix order * Creation and modification time * Implement S3 file and directory * Implement filesystem root * Read bytes * Use a different caching approach * Deptry DEP001 * Change S3 list_contents implementation * Add boto3 as an extra dep * Simpler interface * Use fsspec * Start abstracting away a `FileStream` * Move implementation upstream * Use preferred fsspec abstraction * Use fsspec to list directory contents * Test incremental extraction * Fix types * Clean up * Increase coverage * Move tap logic upstream * Fix imports * Add no cover comment * Configure filesystem name * Update settings docs * CSV-specific settings * Test custom delimiter * Implement schema discovery * Add pragma: no cover * Move ReadMode * Implement FTP and SFTP configurations * Make deptry happy * Link to docs * Update pyproject.toml * Implement primary keys
- Loading branch information
1 parent
5c68b09
commit 1442536
Showing
12 changed files
with
2,468 additions
and
1,248 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,51 +1,51 @@ | ||
| id,first_name,last_name,email,ip_address | ||
| 1,Tobye,Tallach,ttallach0@icq.com,159.80.54.64 | ||
| 2,Bret,Auchterlonie,bauchterlonie1@t-online.de,63.179.228.179 | ||
| 3,Chester,Leban,cleban2@arstechnica.com,109.23.123.220 | ||
| 4,Weston,Venny,wvenny3@about.com,80.78.0.69 | ||
| 5,Alejoa,Hassen,ahassen4@narod.ru,193.70.126.231 | ||
| 6,Otes,Ioselevich,oioselevich5@pen.io,55.238.240.160 | ||
| 7,Dolley,Mc Ilwrick,dmcilwrick6@vistaprint.com,225.224.151.67 | ||
| 8,Cliff,Druitt,cdruitt7@earthlink.net,216.35.85.142 | ||
| 9,Alfreda,Parysiak,aparysiak8@google.com.br,234.124.93.69 | ||
| 10,Alfonso,Wotherspoon,awotherspoon9@cafepress.com,34.94.1.132 | ||
| 11,Jemmy,Gavriel,jgavriela@wikispaces.com,69.13.142.245 | ||
| 12,Ezechiel,Binion,ebinionb@princeton.edu,85.203.127.191 | ||
| 13,Burk,Blowfelde,bblowfeldec@theglobeandmail.com,74.133.42.177 | ||
| 14,Danette,Brealey,dbrealeyd@tumblr.com,249.85.157.243 | ||
| 15,Brent,Collcutt,bcollcutte@twitpic.com,68.202.67.52 | ||
| 16,Filbert,Wane,fwanef@mozilla.org,51.190.146.189 | ||
| 17,Amory,Brewers,abrewersg@sciencedirect.com,147.155.225.194 | ||
| 18,Giraud,Reen,greenh@themeforest.net,134.254.177.66 | ||
| 19,Burtie,Siebert,bsieberti@reuters.com,47.194.48.217 | ||
| 20,Adam,Maddick,amaddickj@theguardian.com,165.16.248.228 | ||
| 21,Callean,Vernall,cvernallk@google.pl,243.145.198.197 | ||
| 22,Olympie,Itzakovitz,oitzakovitzl@addthis.com,40.55.240.15 | ||
| 23,Jacky,Emney,jemneym@cbslocal.com,216.72.80.81 | ||
| 24,Isidoro,Novello,inovellon@symantec.com,153.171.11.150 | ||
| 25,Kora,Liversedge,kliversedgeo@godaddy.com,131.126.97.242 | ||
| 26,Salaidh,McMenamie,smcmenamiep@yolasite.com,228.231.31.219 | ||
| 27,Corey,Dowdeswell,cdowdeswellq@a8.net,203.73.30.64 | ||
| 28,Brodie,Holwell,bholwellr@china.com.cn,64.200.225.25 | ||
| 29,Trudey,Ungerer,tungerers@friendfeed.com,122.82.88.41 | ||
| 30,Doralin,Maxted,dmaxtedt@photobucket.com,37.78.14.199 | ||
| 31,Maurie,Marklin,mmarklinu@gizmodo.com,22.181.178.6 | ||
| 32,Hermann,Voase,hvoasev@gnu.org,155.126.157.84 | ||
| 33,Fanchette,Callaway,fcallawayw@naver.com,121.161.80.246 | ||
| 34,Sara-ann,Birdall,sbirdallx@hexun.com,35.64.166.83 | ||
| 35,Harriot,Clipsham,hclipshamy@redcross.org,78.103.253.219 | ||
| 36,Bonita,Woolway,bwoolwayz@artisteer.com,70.114.50.135 | ||
| 37,Arleyne,MacComiskey,amaccomiskey10@oracle.com,80.22.221.216 | ||
| 38,Ethelbert,Covill,ecovill11@blogs.com,52.66.186.124 | ||
| 39,Irita,Knee,iknee12@skyrock.com,239.247.34.120 | ||
| 40,Naoma,Janca,njanca13@washington.edu,189.63.152.60 | ||
| 41,Kayne,Mizzen,kmizzen14@jalbum.net,84.133.236.10 | ||
| 42,Estell,Stuckford,estuckford15@nationalgeographic.com,246.168.153.22 | ||
| 43,Larine,Stack,lstack16@bravesites.com,197.176.195.68 | ||
| 44,Rikki,Newbold,rnewbold17@omniture.com,27.245.43.243 | ||
| 45,Romonda,Charer,rcharer18@ifeng.com,137.144.236.93 | ||
| 46,Letizia,Monksfield,lmonksfield19@ebay.com,209.47.5.147 | ||
| 47,Sinclare,McAreavey,smcareavey1a@cnn.com,251.250.216.206 | ||
| 48,Athene,Haysham,ahaysham1b@ifeng.com,81.227.231.240 | ||
| 49,Gale,Tracy,gtracy1c@squidoo.com,93.138.226.205 | ||
| 50,Dareen,O'Shields,doshields1d@nymag.com,27.226.127.240 | ||
| id first_name last_name email ip_address | ||
| 1 Tobye Tallach ttallach0@icq.com 159.80.54.64 | ||
| 2 Bret Auchterlonie bauchterlonie1@t-online.de 63.179.228.179 | ||
| 3 Chester Leban cleban2@arstechnica.com 109.23.123.220 | ||
| 4 Weston Venny wvenny3@about.com 80.78.0.69 | ||
| 5 Alejoa Hassen ahassen4@narod.ru 193.70.126.231 | ||
| 6 Otes Ioselevich oioselevich5@pen.io 55.238.240.160 | ||
| 7 Dolley Mc Ilwrick dmcilwrick6@vistaprint.com 225.224.151.67 | ||
| 8 Cliff Druitt cdruitt7@earthlink.net 216.35.85.142 | ||
| 9 Alfreda Parysiak aparysiak8@google.com.br 234.124.93.69 | ||
| 10 Alfonso Wotherspoon awotherspoon9@cafepress.com 34.94.1.132 | ||
| 11 Jemmy Gavriel jgavriela@wikispaces.com 69.13.142.245 | ||
| 12 Ezechiel Binion ebinionb@princeton.edu 85.203.127.191 | ||
| 13 Burk Blowfelde bblowfeldec@theglobeandmail.com 74.133.42.177 | ||
| 14 Danette Brealey dbrealeyd@tumblr.com 249.85.157.243 | ||
| 15 Brent Collcutt bcollcutte@twitpic.com 68.202.67.52 | ||
| 16 Filbert Wane fwanef@mozilla.org 51.190.146.189 | ||
| 17 Amory Brewers abrewersg@sciencedirect.com 147.155.225.194 | ||
| 18 Giraud Reen greenh@themeforest.net 134.254.177.66 | ||
| 19 Burtie Siebert bsieberti@reuters.com 47.194.48.217 | ||
| 20 Adam Maddick amaddickj@theguardian.com 165.16.248.228 | ||
| 21 Callean Vernall cvernallk@google.pl 243.145.198.197 | ||
| 22 Olympie Itzakovitz oitzakovitzl@addthis.com 40.55.240.15 | ||
| 23 Jacky Emney jemneym@cbslocal.com 216.72.80.81 | ||
| 24 Isidoro Novello inovellon@symantec.com 153.171.11.150 | ||
| 25 Kora Liversedge kliversedgeo@godaddy.com 131.126.97.242 | ||
| 26 Salaidh McMenamie smcmenamiep@yolasite.com 228.231.31.219 | ||
| 27 Corey Dowdeswell cdowdeswellq@a8.net 203.73.30.64 | ||
| 28 Brodie Holwell bholwellr@china.com.cn 64.200.225.25 | ||
| 29 Trudey Ungerer tungerers@friendfeed.com 122.82.88.41 | ||
| 30 Doralin Maxted dmaxtedt@photobucket.com 37.78.14.199 | ||
| 31 Maurie Marklin mmarklinu@gizmodo.com 22.181.178.6 | ||
| 32 Hermann Voase hvoasev@gnu.org 155.126.157.84 | ||
| 33 Fanchette Callaway fcallawayw@naver.com 121.161.80.246 | ||
| 34 Sara-ann Birdall sbirdallx@hexun.com 35.64.166.83 | ||
| 35 Harriot Clipsham hclipshamy@redcross.org 78.103.253.219 | ||
| 36 Bonita Woolway bwoolwayz@artisteer.com 70.114.50.135 | ||
| 37 Arleyne MacComiskey amaccomiskey10@oracle.com 80.22.221.216 | ||
| 38 Ethelbert Covill ecovill11@blogs.com 52.66.186.124 | ||
| 39 Irita Knee iknee12@skyrock.com 239.247.34.120 | ||
| 40 Naoma Janca njanca13@washington.edu 189.63.152.60 | ||
| 41 Kayne Mizzen kmizzen14@jalbum.net 84.133.236.10 | ||
| 42 Estell Stuckford estuckford15@nationalgeographic.com 246.168.153.22 | ||
| 43 Larine Stack lstack16@bravesites.com 197.176.195.68 | ||
| 44 Rikki Newbold rnewbold17@omniture.com 27.245.43.243 | ||
| 45 Romonda Charer rcharer18@ifeng.com 137.144.236.93 | ||
| 46 Letizia Monksfield lmonksfield19@ebay.com 209.47.5.147 | ||
| 47 Sinclare McAreavey smcareavey1a@cnn.com 251.250.216.206 | ||
| 48 Athene Haysham ahaysham1b@ifeng.com 81.227.231.240 | ||
| 49 Gale Tracy gtracy1c@squidoo.com 93.138.226.205 | ||
| 50 Dareen O'Shields doshields1d@nymag.com 27.226.127.240 |
Oops, something went wrong.