Local peer discovery

Peergos can now discover peers on the local network (LAN) using mdns. This means two peers on the same LAN can friend each other and share files etc entirely offline.

Features:

• local peer discovery using mdns
• launch recommended-app for secret links to supported formats (docx, clsx etc.)
• make server configurable in writable-link generator cli

Optimisations:

• cache pointer targets
• improve newsfeed performance

Fixes:

• bulk mirror of raw blocks
• offline login
• fix stay logged in with MFA enabled
• remove any cached login data if MFA is enabled

Mirrored at https://peergos.net/public/peergos/releases

Release commit:
web-ui/f6788b5a3366148e21dcf70d56a5a735e4c6243b

60x faster mirroring, chat API

The initial pki sync is now 60x faster (down from 10min to 10s). The pki sync also happens in the background now. There is a new chat api enabling fully fledged 3rd party chat apps.

Features:

• Extended chat API for apps
• Background pki sync
• Show profile picture in drive header
• Improved newsfeed design
• Thumbnails displayed in picker dialogue
• Directly launch specific apps from search results
• Additional app permission to retrieve profile photos

Optimisations:

• Mirroring data is about 60X faster now

Fixes:

• Use direct S3 blockstore when available
• Fix bulk chat invites
• Improve S3 error handling

Mirrored at https://peergos.net/public/peergos/releases

Release commit:
web-ui/04354df8a322502916e792a496b1e7604c84a9a0

Server identity rotation and internationalisation

Peergos v0.16.0

Servers can now rotate their identity keypair independently of the users on that server. Clients verify the rotation using signed IPNS records if they update their host, which are periodically republished. Internationalisation support is included with simplified Chinese supported already.

Features

• remove 2 GB limit for size of zipped folder that can be downloaded
• server identity rotation (and CLI)
• select all/none

Optimisations

• make GC about 100x more scalable by storing reachability and block list on disk instead of ram

Fixes

• fix native image builds
• handle OPFS returning empty blocks
• remove excess space at bottom of login page
• fix race condition with repeated updateSocial calls during login
• fix multi-select in UI
• fix occasional failures to resolve a peerid in dht
• miscellaneous web ui improvements
• context menu dark mode

Mirrored at https://peergos.net/public/peergos/releases

Release commit:
web-ui/512532b3a2e280f5d15e0b89bc8a050f050b76f4

Latex support, 15x less bandwidth

Peergos v0.15.0

Large performance improvements after switch to Nabu. 15x reduction in baseline bandwidth usage. Now ~12KB/s is typical. Also 60% less CPU usage. The size of peergos.js has also been halved by a bug fix in jpack!

Features:

• Latex support in notes app
• Latex support in markdown app
• Support UTF-8 in text mimetype detection
• Usage CLI for server operators
• Authed usage call

Optimisations:

• Don't query blockstore when calculating usage for users that haven't changed
• Add local cbor block cache for S3 to make champ gets 10x faster
• 2x less RAM and bandwidth usage in Kademlia
• Huge reduction to kademlia routing table size by not adding non-server peers
• Don't call identify or kademlia on incoming connections for peers we already know
• Reduce baseline bandwidth usage from 1.5Mb/s to 0.1Mb/s by switching to a new bitswap protocol ID
• Fix Jpack to halve size of compiled peergos.js (2.1 MB to 1.1 MB)

Fixes:

• Handle S3 internal errors as rate limits
• Correct usage update during migration to new node
• Improve information for GC errors
• Make GC tolerant to non existent blocks
• Handle .apk mimetype
• Treat http connection refused as a rate limit
• Darkmode fix for sub pages in md viewer
• Fix opening depth 2 sub-pages in md viewer

Published at https://peergos.net/public/peergos/releases

Release commit:
web-ui/ac4bba146e5faad4f98ab9ca1b5aebd330823667

Notes app + bug fixes

A new app and a few bug fixes from the previous major rewrite in v0.14.0.

New apps:

• notes

Features:

• ipfs metrics for S3 calls, bitswap and kademlia bandwidth
• decimal storage sizes

Optimisations:

• cache peer id in AuthedStorage
• inline block request authoriser (no separate allow listener)
• block aggressive bitswap peers
• don't retrieve blocks to auth a bitswap request for them

Fixes:

• timezone for block auth generation
• use bare peer id in bat auth
• always persist generated ipfs identity
• make sure block metadata db is populted for file blockstores also
• handle empty block files
• filter blocks of wrong type in listing blockstore

Release commit:
web-ui/163f59fcbbff6e3cf2814de985b7bba5c0585e97

Java IPFS implementation (Nabu), block metadata db, versioned S3 buckets.

This is an exciting new release that means we finally control our whole stack as we switch to a Java implementation of IPFS (that we wrote). We introduce a blockmetadata store that stores for each cid the size, links and BATs. This massively speeds up GC, space usage updates, and read authorizations.

Features:

• Switch from Golang based ipfs-nucleus to Java based Nabu
• Support versioned S3 buckets
• Introduce block metadata db to massively speed up many operations
• infini filter (infinitely expandable bloom filter)
• only use authWrites for raw blocks, and send the BATs for these
• store BATs in block metadata store

Privacy improvements

• don't broadcast block retrievals to DHT or bitswap
• don't provide blocks to DHT unless we are running a mirror (also reduces CPU and bandwidth)

Optimisations:

• 10x less bandwidth usage when idle
• faster peer dials to get blocks
• less CPU and RAM usage
• inline p2p block get authorization
• block repeated block get attempts with invalid auth
• huge speed up to cacheless delete (e.g. in incognito mode)
• use owner to lookup peer to retrieve block from in the common case
• improve sign up oplog from 10 pointer updates and 29 blocks to, 2 pointer updates and 10 blocks
• upgrade HTTP client in JavaPoster

Fixes:

• timezone issue in BAT verification
• fix handling of writable secret links
• Bulk delete on B2 (which is not S3 compliant)
• CAS exception making social post
• support Ed25519 PKI peer id
• event loading in calendar

Release commit:
web-ui/59c4fa5820dd7964a70d992c7bec97ff252e98ea

v0.13.0

Two factor auth (2FA), Multi-file-select, new apps

Features:

• 2FA including TOTP and webauthn
• select multiple files/folders to {delete, download, cut, copy, paste}
• enable quic, noise
• require token to signup if not only listening on localhost
• support HEIC and JPEGXL image mimetypes
• require Java 17
• build docker image
• add delete chat and setTitle to chat API
• human readable file sizes
• new and improved gallery with media controls and prefetch
• app grid on launcher page
• file picker api for apps
• export to PDF in paintz, ck-editor and tui-markdown editor apps
• app to convert html to docx
• new draw.io app

Optimisations:

• download chunks in parallel (much faster downloads)
• make bloom adds async
• bulk block writes for small blocks (speed up deletes, uploads)
• parallelise block puts to S3 in bulk write
• bulk delete
• add OPFS (Origin Private File System) cache for blocks in browser

Fixes:

• fix svg mimetype case
• allow wildcard listen addresses (fixes docker)
• fix immediate login after changing password in browser
• many miscellaneous fixes

App-store (recommended apps), full-text search

Peergos v0.12.0

App-store (recommended apps), full-text search

Features:

• full text search of files
• implement a bulk upload progress bar, rather than one per file
• add "app store" for easy installation of recommended apps
• allow secret links to open apps
• enable noise security protocol for future compatibility
• support available apps in the newsfeed directly
• show available apps for opening a file in context menu

Optimisations:

• make GC 60x faster using block metadata store
• make usage calculation 60x faster using block metadata store
• increase buffer size when streaming video in browser
• improve offline champ lookups

Fixes:

• miscellaneous reliability fixes
• mirroring to S3 based node
• legacy directory modification
• granting write access
• better offline detection
• timezone handling in recurring calendar events
• better concurrent file modification handling

Download binaries from https://peergos.net/public/peergos/releases

Offline login and viewing of files

Peergos v0.11.0

Offline login and viewing of files.

Features:

• include retrying mutable pointers, and extra feedback during CLI upload
• add chat api for peergos apps
• support local app dev via a proxy
• support http HEAD and OPTIONS in applications
• add offline block and pointer cache in browser (using indexeddb)
• add ability for apps to ask user to save a file
• peergos is now a Progressive Web App (installable from browser on sensible OSes)
• 100% offline login and viewing of recent files

Optimisations:

• reduce S3 read auth bandwidth by 80X to 10,000X
• parallelise usage calculation
• constrain CPU usage of ipfs-nucleus
• big speedups across the board by always using a buffered network access
• speed up news feed building and retrieving shared files by hoisting pointer gets
• halve S3 read latency from browser (no CORS preflight)
• make bulk import of calendars 30x faster
• use bloom filter in S3 case to reduce request rate by 1000X

Fixes:

• auto open for public link to a single file
• paste file bugs
• uploading huge videos through Java CLI
• include https support in native images
• support dash encoded mp4 videos
• fix space usage growth - now uploading and deleting a file returns you to prior usage
• add a transaction store GC
• fix build on windows
• allow empty summary in calendar import
• make calendar import idempotent
• fix duplicate progress bar bug during multiple file upload
• handle S3 rate limits during signup
• delay updating progress bars until blocks have been flushed from buffered storage
• handle video thumbnail error causing upload hangs
• htmlviewer deep links

Download binaries from: https://peergos.net/public/peergos/releases

Application sandbox and private websites

Changes:

• implement htmlviewer app to view private websites
• implement app sandbox using generated hostnames, COOP, COEP, CORP and CSP.
• add app launcher, and folder shortcuts
• implement needed calls for FUSE mount on MacOS
• allow unblocking users
• auto update current day in calendar
• add creation time to search results
• speed up building newsfeed
• Speed up file deletion by 9X
• Implement folder delete in shell rm command

Fixes:

• thumbnail aware upload progress
• make feed updates atomic
• handle tabs in ascii mimetype detection
• fix closing apps in secret link that auto opens
• fix creating a new todo board in legacy accounts

Release commit:
web-ui/8a5614890aaac8da620d0f2d14ffebcdaa883780