Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: replace node buffers with uint8arrays #67

Merged
merged 3 commits into from
Aug 14, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .aegir.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
'use strict'

module.exports = {
webpack: {
node: {
path: true,

Buffer: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've updated the file with comments about which modules require the node libs

}
}
}
38 changes: 24 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# IPNS
# IPNS <!-- omit in toc -->

[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://protocol.ai)
[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)
Expand All @@ -12,20 +12,30 @@

This module contains all the necessary code for creating, understanding and validating IPNS records.

## Lead Maintainer
## Lead Maintainer <!-- omit in toc -->

[Vasco Santos](https://github.com/vasco-santos).

## Table of Contents
## Table of Contents <!-- omit in toc -->

- [Install](#install)
- [Usage](#usage)
- [Create Record](#create-record)
- [Validate Record](#validate-record)
- [Embed public key to record](#embed-public-key-to-record)
- [Extract public key from record](#extract-public-key-from-record)
- [Datastore key](#datastore-key)
- [Create record](#create-record)
- [Validate record](#validate-record)
- [Embed public key to record](#embed-public-key-to-record)
- [Extract public key from record](#extract-public-key-from-record)
- [Datastore key](#datastore-key)
- [Marshal data with proto buffer](#marshal-data-with-proto-buffer)
- [Unmarshal data from proto buffer](#unmarshal-data-from-proto-buffer)
- [Validator](#validator)
- [API](#api)
- [Create record](#create-record-1)
- [Validate record](#validate-record-1)
- [Datastore key](#datastore-key-1)
- [Marshal data with proto buffer](#marshal-data-with-proto-buffer-1)
- [Unmarshal data from proto buffer](#unmarshal-data-from-proto-buffer-1)
- [Embed public key to record](#embed-public-key-to-record-1)
- [Extract public key from record](#extract-public-key-from-record-1)
- [Namespace](#namespace)
- [Contribute](#contribute)
- [License](#license)

Expand Down Expand Up @@ -136,13 +146,13 @@ Create an IPNS record for being stored in a protocol buffer.
- `lifetime` (string): lifetime of the record (in milliseconds).

Returns a `Promise` that resolves to an object with the entry's properties eg:

```js
{
value: '/ipfs/QmWEekX7EZLUd9VXRNMRXW3LXe4F6x7mB8oPxY5XLptrBq',
signature: Buffer,
value: Uint8Array,
signature: Uint8Array,
validityType: 0,
validity: '2018-06-27T14:49:14.074000000Z',
validity: Uint8Array,
sequence: 2
}
```
Expand Down Expand Up @@ -189,7 +199,7 @@ const data = ipns.unmarshal(storedData)

Returns the entry data structure after being serialized.

- `storedData` (Buffer): ipns entry record serialized.
- `storedData` (Uint8Array): ipns entry record serialized.

#### Embed public key to record

Expand Down
14 changes: 7 additions & 7 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,26 +33,26 @@
},
"homepage": "https://github.com/ipfs/js-ipns#readme",
"dependencies": {
"buffer": "^5.6.0",
"debug": "^4.1.1",
"err-code": "^2.0.0",
"interface-datastore": "^1.0.2",
"interface-datastore": "^2.0.0",
"libp2p-crypto": "^0.17.1",
"multibase": "^3.0.0",
"multihashes": "^3.0.1",
"peer-id": "^0.13.6",
"protons": "^1.0.1",
"timestamp-nano": "^1.0.0"
"peer-id": "^0.14.0",
"protons": "^2.0.0",
"timestamp-nano": "^1.0.0",
"uint8arrays": "^1.1.0"
},
"devDependencies": {
"aegir": "^23.0.0",
"aegir": "^25.1.0",
"chai": "^4.2.0",
"chai-bytes": "~0.1.2",
"chai-string": "^1.5.0",
"dirty-chai": "^2.0.1",
"ipfs": "^0.49.0",
"ipfs-http-client": "^46.0.0",
"ipfsd-ctl": "^4.0.1"
"ipfsd-ctl": "^6.0.0"
},
"contributors": [
"Vasco Santos <vasco.santos@moxy.studio>",
Expand Down
43 changes: 26 additions & 17 deletions src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ const multihash = require('multihashes')
const errCode = require('err-code')
const { Buffer } = require('buffer')
const multibase = require('multibase')
const uint8ArrayFromString = require('uint8arrays/from-string')
const uint8ArrayToString = require('uint8arrays/to-string')
const uint8ArrayConcat = require('uint8arrays/concat')

const debug = require('debug')
const log = debug('jsipns')
Expand All @@ -25,7 +28,7 @@ const namespace = '/ipns/'
* IPNS entry
* @typedef {Object} IpnsEntry
* @property {string} value - value to be stored in the record
* @property {Buffer} signature - signature of the record
* @property {Uint8Array} signature - signature of the record
* @property {number} validityType - Type of validation being used
* @property {string} validity - expiration datetime for the record in RFC3339 format
* @property {number} sequence - number representing the version of the record
Expand Down Expand Up @@ -67,10 +70,10 @@ const _create = async (privateKey, value, seq, isoValidity, validityType) => {
const signature = await sign(privateKey, value, validityType, isoValidity)

const entry = {
value: value,
value: uint8ArrayFromString(value),
signature: signature,
validityType: validityType,
validity: isoValidity,
validity: uint8ArrayFromString(isoValidity),
sequence: seq
}

Expand Down Expand Up @@ -106,7 +109,7 @@ const validate = async (publicKey, entry) => {
let validityDate

try {
validityDate = parseRFC3339(validity.toString())
validityDate = parseRFC3339(uint8ArrayToString(validity))
} catch (e) {
log.error('unrecognized validity format (not an rfc3339 format)')
throw errCode(new Error('unrecognized validity format (not an rfc3339 format)'), ERRORS.ERR_UNRECOGNIZED_FORMAT)
Expand Down Expand Up @@ -193,7 +196,7 @@ const extractPublicKey = (peerId, entry) => {
if (entry.pubKey) {
let pubKey
try {
pubKey = crypto.keys.unmarshalPublicKey(entry.pubKey)
pubKey = crypto.keys.unmarshalPublicKey(Buffer.from(entry.pubKey))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't we replacing the Buffers everywhere?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The underlying crypto libraries do not understand Uint8Arrays and require Buffers :(

But! The user should be able to pass Uint8Arrays to our modules without them exploding.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be updated to use libp2p-crypto@0.18.0, it does use Uint8Array now

} catch (err) {
log.error(err)
throw err
Expand All @@ -214,7 +217,7 @@ const rawStdEncoding = (key) => multibase.encode('base32', key).toString().slice
* Get key for storing the record locally.
* Format: /ipns/${base32(<HASH>)}
*
* @param {Buffer} key peer identifier object.
* @param {Uint8Array} key peer identifier object.
* @returns {string}
*/
const getLocalKey = (key) => new Key(`/ipns/${rawStdEncoding(key)}`)
Expand All @@ -223,18 +226,18 @@ const getLocalKey = (key) => new Key(`/ipns/${rawStdEncoding(key)}`)
* Get key for sharing the record in the routing mechanism.
* Format: ${base32(/ipns/<HASH>)}, ${base32(/pk/<HASH>)}
*
* @param {Buffer} pid peer identifier represented by the multihash of the public key as Buffer.
* @param {Uint8Array} pid peer identifier represented by the multihash of the public key as Uint8Array.
* @returns {Object} containing the `nameKey` and the `ipnsKey`.
*/
const getIdKeys = (pid) => {
const pkBuffer = Buffer.from('/pk/')
const ipnsBuffer = Buffer.from('/ipns/')
const pkBuffer = uint8ArrayFromString('/pk/')
const ipnsBuffer = uint8ArrayFromString('/ipns/')

return {
routingPubKey: new Key(Buffer.concat([pkBuffer, pid]), false), // Added on https://github.com/ipfs/js-ipns/pull/8#issue-213857876 (pkKey will be deprecated in a future release)
pkKey: new Key(rawStdEncoding(Buffer.concat([pkBuffer, pid]))),
routingKey: new Key(Buffer.concat([ipnsBuffer, pid]), false), // Added on https://github.com/ipfs/js-ipns/pull/6#issue-213631461 (ipnsKey will be deprecated in a future release)
ipnsKey: new Key(rawStdEncoding(Buffer.concat([ipnsBuffer, pid])))
routingPubKey: new Key(uint8ArrayConcat([pkBuffer, pid]), false), // Added on https://github.com/ipfs/js-ipns/pull/8#issue-213857876 (pkKey will be deprecated in a future release)
pkKey: new Key(rawStdEncoding(uint8ArrayConcat([pkBuffer, pid]))),
routingKey: new Key(uint8ArrayConcat([ipnsBuffer, pid]), false), // Added on https://github.com/ipfs/js-ipns/pull/6#issue-213631461 (ipnsKey will be deprecated in a future release)
ipnsKey: new Key(rawStdEncoding(uint8ArrayConcat([ipnsBuffer, pid])))
}
}

Expand Down Expand Up @@ -263,11 +266,17 @@ const getValidityType = (validityType) => {

// Utility for creating the record data for being signed
const ipnsEntryDataForSig = (value, validityType, validity) => {
const valueBuffer = Buffer.from(value)
const validityTypeBuffer = Buffer.from(getValidityType(validityType))
const validityBuffer = Buffer.from(validity)
if (!(value instanceof Uint8Array)) {
value = uint8ArrayFromString(value)
}

if (!(validity instanceof Uint8Array)) {
validity = uint8ArrayFromString(validity)
}

const validityTypeBuffer = uint8ArrayFromString(getValidityType(validityType))

return Buffer.concat([valueBuffer, validityBuffer, validityTypeBuffer])
return uint8ArrayConcat([value, validity, validityTypeBuffer])
}

// Utility for extracting the public key from a peer-id
Expand Down
20 changes: 10 additions & 10 deletions test/index.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ const chai = require('chai')
const dirtyChai = require('dirty-chai')
const chaiBytes = require('chai-bytes')
const chaiString = require('chai-string')
const { Buffer } = require('buffer')
const expect = chai.expect
chai.use(dirtyChai)
chai.use(chaiBytes)
chai.use(chaiString)
const { toB58String } = require('multihashes')
const uint8ArrayFromString = require('uint8arrays/from-string')

const ipfs = require('ipfs')
const ipfsHttpClient = require('ipfs-http-client')
Expand Down Expand Up @@ -56,7 +56,7 @@ describe('ipns', function () {

const entry = await ipns.create(rsa, cid, sequence, validity)
expect(entry).to.deep.include({
value: cid,
value: uint8ArrayFromString(cid),
sequence: sequence
})
expect(entry).to.have.a.property('validity')
Expand All @@ -73,7 +73,7 @@ describe('ipns', function () {

await ipns.validate(rsa.public, entry)
expect(entry).to.have.a.property('validity')
expect(entry.validity).to.equal('2033-05-18T03:33:20.000000000Z')
expect(entry.validity).to.equalBytes(uint8ArrayFromString('2033-05-18T03:33:20.000000000Z'))
})

it('should create an ipns record and validate it correctly', async () => {
Expand Down Expand Up @@ -129,8 +129,8 @@ describe('ipns', function () {
const marshalledData = ipns.marshal(entryDataCreated)
const unmarshalledData = ipns.unmarshal(marshalledData)

expect(entryDataCreated.value).to.equal(unmarshalledData.value.toString())
expect(entryDataCreated.validity).to.equal(unmarshalledData.validity.toString())
expect(entryDataCreated.value).to.equalBytes(unmarshalledData.value)
expect(entryDataCreated.validity).to.equalBytes(unmarshalledData.validity)
expect(entryDataCreated.validityType).to.equal(unmarshalledData.validityType)
expect(entryDataCreated.signature).to.equalBytes(unmarshalledData.signature)
expect(entryDataCreated.sequence).to.equal(unmarshalledData.sequence)
Expand Down Expand Up @@ -166,7 +166,7 @@ describe('ipns', function () {

keys.forEach(key => {
const { routingKey } = ipns.getIdKeys(fromB58String(key))
const id = toB58String(routingKey.toBuffer().slice(ipns.namespaceLength))
const id = toB58String(routingKey.uint8Array().subarray(ipns.namespaceLength))

expect(id).to.equal(key)
})
Expand Down Expand Up @@ -206,7 +206,7 @@ describe('ipns', function () {
const entry = await ipns.create(rsa, cid, sequence, validity)

const marshalledData = ipns.marshal(entry)
const key = Buffer.from(`/ipns/${ipfsId.id}`)
const key = uint8ArrayFromString(`/ipns/${ipfsId.id}`)

try {
await ipns.validator.validate(marshalledData, key)
Expand Down Expand Up @@ -235,7 +235,7 @@ describe('ipns', function () {
await ipns.embedPublicKey(rsa.public, entry)

const marshalledData = ipns.marshal(entry)
const key = Buffer.from(`/ipns/${ipfsId.id}`)
const key = uint8ArrayFromString(`/ipns/${ipfsId.id}`)

const valid = await ipns.validator.validate(marshalledData, key)
expect(valid).to.equal(true)
Expand All @@ -249,9 +249,9 @@ describe('ipns', function () {
await ipns.embedPublicKey(rsa.public, entry)

// corrupt the record by changing the value to random bytes
entry.value = crypto.randomBytes(46).toString()
entry.value = crypto.randomBytes(46)
const marshalledData = ipns.marshal(entry)
const key = Buffer.from(`/ipns/${ipfsId.id}`)
const key = uint8ArrayFromString(`/ipns/${ipfsId.id}`)

try {
await ipns.validator.validate(marshalledData, key)
Expand Down