Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support snappy format #215

Merged
merged 1 commit into from
Dec 8, 2021
Merged

support snappy format #215

merged 1 commit into from
Dec 8, 2021

Conversation

figsoda
Copy link
Member

@figsoda figsoda commented Dec 7, 2021

closes #202

Also removed remaining mentions of incorrect lz extensions that wasn't removed in #198

@marcospb19
Copy link
Member

Nice, does the Snappy format has any MIME bytes to be add to the tests?

@figsoda
Copy link
Member Author

figsoda commented Dec 7, 2021

yes, application/x-snappy-framed, but infer doesn't support it
file-format supports it but is not nearly as popular, maybe we can try that
didn't check but magic (libmagic binding) might support it as welll

@marcospb19
Copy link
Member

For reference, we have some MIME checks implemented here:

ouch/src/utils/fs.rs

Lines 65 to 121 in bafc2d3

pub fn try_infer_extension(path: &Path) -> Option<Extension> {
fn is_zip(buf: &[u8]) -> bool {
buf.len() >= 3
&& buf[..=1] == [0x50, 0x4B]
&& (buf[2..=3] == [0x3, 0x4] || buf[2..=3] == [0x5, 0x6] || buf[2..=3] == [0x7, 0x8])
}
fn is_tar(buf: &[u8]) -> bool {
buf.len() > 261 && buf[257..=261] == [0x75, 0x73, 0x74, 0x61, 0x72]
}
fn is_gz(buf: &[u8]) -> bool {
buf.starts_with(&[0x1F, 0x8B, 0x8])
}
fn is_bz2(buf: &[u8]) -> bool {
buf.starts_with(&[0x42, 0x5A, 0x68])
}
fn is_xz(buf: &[u8]) -> bool {
buf.starts_with(&[0xFD, 0x37, 0x7A, 0x58, 0x5A, 0x00])
}
fn is_lz4(buf: &[u8]) -> bool {
buf.starts_with(&[0x04, 0x22, 0x4D, 0x18])
}
fn is_zst(buf: &[u8]) -> bool {
buf.starts_with(&[0x28, 0xB5, 0x2F, 0xFD])
}
let buf = {
let mut buf = [0; 270];
// Error cause will be ignored, so use std::fs instead of fs_err
let result = std::fs::File::open(&path).map(|mut file| file.read(&mut buf));
// In case of file open or read failure, could not infer a extension
if result.is_err() {
return None;
}
buf
};
use crate::extension::CompressionFormat::*;
if is_zip(&buf) {
Some(Extension::new(&[Zip], "zip"))
} else if is_tar(&buf) {
Some(Extension::new(&[Tar], "tar"))
} else if is_gz(&buf) {
Some(Extension::new(&[Gzip], "gz"))
} else if is_bz2(&buf) {
Some(Extension::new(&[Bzip], "bz2"))
} else if is_xz(&buf) {
Some(Extension::new(&[Lzma], "xz"))
} else if is_lz4(&buf) {
Some(Extension::new(&[Lz4], "lz4"))
} else if is_zst(&buf) {
Some(Extension::new(&[Zstd], "zst"))
} else {
None
}
}

So, this could probably be reused in our tests?

If you wish to, I can merge this as it is, and then you can create an issue for covering the rest of it.

Or, you can try to solve everything in this PR.

@figsoda
Copy link
Member Author

figsoda commented Dec 8, 2021

I thought about reusing it by checking the warnings generated by the program, but #220 (just opened btw) might cause tests to fail

I don't plan on addressing this in this pull request

@marcospb19 marcospb19 merged commit fddc796 into ouch-org:master Dec 8, 2021
@figsoda figsoda deleted the snappy branch December 8, 2021 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support snappy format
2 participants