forked from plougher/squashfs-tools
-
Notifications
You must be signed in to change notification settings - Fork 0
/
USAGE
1287 lines (1011 loc) · 51.3 KB
/
USAGE
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
SQUASHFS - A squashed read-only filesystem for Linux
Copyright 2002-2022 Phillip Lougher <phillip@squashfs.org.uk>
Released under the GPL licence (version 2 or later).
Welcome to Squashfs-tools. Please read the CHANGES and README-4.5 or
README-4.5.1 files for the added features and improvements. This guide has not
been updated to include these new features, and so that should be read instead
for these.
Squashfs is a highly compressed read-only filesystem for Linux.
It uses either gzip/xz/lzo/lz4/zstd compression to compress both files, inodes
and directories. Inodes in the system are very small and all blocks are
packed to minimise data overhead. Block sizes greater than 4K are supported
up to a maximum of 1Mbytes (default block size 128K).
Squashfs is intended for general read-only filesystem use, for archival
use (i.e. in cases where a .tar.gz file may be used), and in constrained
block device/memory systems (e.g. embedded systems) where low overhead is
needed.
1. SQUASHFS OVERVIEW
--------------------
1. Data, inodes and directories are compressed.
2. Squashfs stores full uid/gids (32 bits), and file creation time.
3. In theory files up to 2^64 bytes are supported. In theory filesystems can
be up to 2^64 bytes.
4. Inode and directory data are highly compacted, and packed on byte
boundaries. Each compressed inode is on average 8 bytes in length
(the exact length varies on file type, i.e. regular file, directory,
symbolic link, and block/char device inodes have different sizes).
5. Squashfs can use block sizes up to 1Mbyte (the default size is 128K).
Using 128K blocks achieves greater compression ratios than the normal
4K block size.
6. File duplicates are detected and removed.
7. Filesystems can be compressed with gzip, xz (lzma2), lzo, lz4
or zstd compression algorithms.
1.1 Introducing reproducible builds
-----------------------------------
Ever since Mksquashfs was parallelised back in 2006, there
has been a certain randomness in how fragments and multi-block
files are ordered in the output filesystem even if the input
remains the same.
This is because the multiple parallel threads can be scheduled
differently between Mksquashfs runs. For example, the thread
given fragment 10 to compress may finish before the thread
given fragment 9 to compress on one run (writing fragment 10
to the output filesystem before fragment 9), but, on the next
run it could be vice-versa. There are many different scheduling
scenarios here, all of which can have a knock on effect causing
different scheduling and ordering later in the filesystem too.
Mkquashfs doesn't care about the ordering of fragments and
multi-block files within the filesystem, as this does not
affect the correctness of the filesystem.
In fact not caring about the ordering, as it doesn't matter, allows
Mksquashfs to run as fast as possible, maximising CPU and I/O
performance.
But, in the last couple of years, Squashfs has become used in
scenarios (cloud etc) where this randomness is causing problems.
Specifically this appears to be where downloaders, installers etc.
try to work out the differences between Squashfs filesystem
updates to minimise the amount of data that needs to transferred
to update an image.
Additionally, in the last couple of years has arisen the notion
of reproducible builds, that is the same source and build
environment etc should be able to (re-)generate identical
output. This is usually for verification and security, allowing
binaries/distributions to be checked for malicious activity.
See https://reproducible-builds.org/ for more information.
Mksquashfs from release 4.4 now generates reproducible images
by default. Images generated by Mksquashfs will be ordered
identically to previous runs if the same input has been supplied,
and the same options used.
1.1.1 Dealing with timestamps
Timestamps embedded in the filesystem will stiil cause differences.
Each new run of Mksquashfs will produce a different mkfs (make filesystem)
timestamp in the super-block. Moreover if any file timestamps have changed
(even if the content hasn't), this will produce a difference.
To prevent timestamps from producing differences, the following
new Mksquashfs options have been added.
1.1.2 -mkfs-time <time>
This option takes a positive time value (which is the number
of seconds since the epoch of 1970-01-01 00:00:00 UTC), and sets
the file system timestamp to that.
Squashfs uses an unsigned 32-bit integer to store time, and the
time given should be in that range.
Obviously you can use the date command to convert dates into
this value, i.e.
% mksquashfs source source.sqsh -mkfs-time $(date +%s -d "Jan 1 2019 19:00")
1.1.3 -all-time <time>
This option takes a positive time value (which is the number
of seconds since the epoch of 1970-01-01 00:00:00 UTC), and sets
the timestamp on all files to that (but not the mkfs time).
1.1.4 environment variable SOURCE_DATE_EPOCH
As an alternative to the above command line options, you can
set the environment variable SOURCE_DATE_EPOCH to a time value.
This value will be used to set the mkfs time. Also any
file timestamps which are after SOURCE_DATE_EPOCH will be
clamped to SOURCE_DATE_EPOCH.
See https://reproducible-builds.org/docs/source-date-epoch/
for more information.
Note: both SOURCE_DATE_EPOCH and the command line options cannot
be used at the same time. They are different ways to do the same thing,
and both have FORCE sematics which mean they can't be over-ridden
elsewhere (otherwise it would defeat the purpose).
1.1.5 -not-reproducible
This option tells Mksquashfs that the files do not have to be
strictly ordered. This will make Mksquashfs behave like version 4.3.
1.2 Extended attributes (xattrs)
--------------------------------
Squashfs filesystems now have extended attribute support. The
extended attribute implementation has the following features:
1. Layout can store up to 2^48 bytes of compressed xattr data.
2. Number of xattrs per inode unlimited.
3. Total size of xattr data per inode 2^48 bytes of compressed data.
4. Up to 4 Gbytes of data per xattr value.
5. Inline and out-of-line xattr values supported for higher performance
in xattr scanning (listxattr & getxattr), and to allow xattr value
de-duplication.
6. Both whole inode xattr duplicate detection and individual xattr value
duplicate detection supported. These can obviously nest, file C's
xattrs can be a complete duplicate of file B, and file B's xattrs
can be a partial duplicate of file A.
7. Xattr name prefix types stored, allowing the redundant "user.", "trusted."
etc. characters to be eliminated and more concisely stored.
8. Support for files, directories, symbolic links, device nodes, fifos
and sockets.
Extended attribute support is in 2.6.35 and later kernels. Filesystems
with extended attributes can be mounted on 2.6.29 and later kernels, the
extended attributes will be ignored with a warning.
2. USING SQUASHFS
-----------------
Squashfs filesystems should be mounted with 'mount' with the filesystem type
'squashfs'. If the filesystem is on a block device, the filesystem can be
mounted directly, e.g.
%mount -t squashfs /dev/sda1 /mnt
Will mount the squashfs filesystem on "/dev/sda1" under the directory "/mnt".
If the squashfs filesystem has been written to a file, the loopback device
can be used to mount it (loopback support must be in the kernel), e.g.
%mount -t squashfs image /mnt -o loop
Will mount the squashfs filesystem in the file "image" under
the directory "/mnt".
3. MKSQUASHFS
-------------
3.1 Mksquashfs options and overview
-----------------------------------
As squashfs is a read-only filesystem, the mksquashfs program must be used to
create populated squashfs filesystems.
SYNTAX:mksquashfs source1 source2 ... dest [options] [-e list of exclude
dirs/files]
Filesystem build options:
-tar read uncompressed tar file from standard in (stdin)
-no-strip act like tar, and do not strip leading directories
from source files
-tarstyle alternative name for -no-strip
-cpiostyle act like cpio, and read files from standard in (stdin)
-cpiostyle0 like -cpiostyle, but filenames are null terminated
-comp <comp> select <comp> compression
Compressors available:
gzip (default)
lzo
lz4
xz
zstd
-b <block_size> set data block to <block_size>. Default 128 Kbytes
Optionally a suffix of K or M can be given to specify
Kbytes or Mbytes respectively
-reproducible build images that are reproducible (default)
-not-reproducible build images that are not reproducible
-mkfs-time <time> set mkfs time to <time> which is an unsigned int
-fstime <time> synonym for mkfs-time
-all-time <time> set all inode times to <time> which is an unsigned int
-no-exports don't make filesystem exportable via NFS (-tar default)
-exports make filesystem exportable via NFS (default)
-no-sparse don't detect sparse files
-no-xattrs don't store extended attributes
-xattrs store extended attributes (default)
-noI do not compress inode table
-noId do not compress the uid/gid table (implied by -noI)
-noD do not compress data blocks
-noF do not compress fragment blocks
-noX do not compress extended attributes
-no-tailends don't pack tail ends into fragments (default)
-tailends pack tail ends into fragments
-no-fragments do not use fragments
-always-use-fragments use fragment blocks for files larger than block size
-no-duplicates do not perform duplicate checking
-no-hardlinks do not hardlink files, instead store duplicates
-all-root make all files owned by root
-root-time <time> set root directory time to <time>
-root-mode <mode> set root directory permissions to octal <mode>
-root-uid <uid> set root directory owner to <uid>
-root-gid <gid> set root directory group to <gid>
-force-uid <uid> set all file uids to <uid>
-force-gid <gid> set all file gids to <gid>
-nopad do not pad filesystem to a multiple of 4K
-keep-as-directory if one source directory is specified, create a root
directory containing that directory, rather than the
contents of the directory
-action <action@expr> evaluate <expr> on every file, and execute <action>
if it returns TRUE
-log-action <act@expr> as above, but log expression evaluation results and
actions performed
-true-action <act@expr> as above, but only log expressions which return TRUE
-false-action <act@exp> as above, but only log expressions which return FALSE
-action-file <file> as action, but read actions from <file>
-log-action-file <file> as -log-action, but read actions from <file>
-true-action-file <f> as -true-action, but read actions from <f>
-false-action-file <f> as -false-action, but read actions from <f>
Filesystem filter options:
-p <pseudo-definition> Add pseudo file definition
-pf <pseudo-file> Add list of pseudo file definitions
Pseudo definitions should be of the format
filename d mode uid gid
filename m mode uid gid
filename b mode uid gid major minor
filename c mode uid gid major minor
filename f mode uid gid command
filename s mode uid gid symlink
filename i mode uid gid [s|f]
filename l filename
filename L pseudo_filename
filename D time mode uid gid
filename M time mode uid gid
filename B time mode uid gid major minor
filename C time mode uid gid major minor
filename F time mode uid gid command
filename S time mode uid gid symlink
filename I time mode uid gid [s|f]
filename R time mode uid gid length offset
-sort <sort_file> sort files according to priorities in <sort_file>. One
file or dir with priority per line. Priority -32768 to
32767, default priority 0
-ef <exclude_file> list of exclude dirs/files. One per line
-wildcards Allow extended shell wildcards (globbing) to be used in
exclude dirs/files
-regex Allow POSIX regular expressions to be used in exclude
dirs/files
-one-file-system Do not cross filesystem boundaries when scanning sources
Filesystem append options:
-noappend do not append to existing filesystem
-root-becomes <name> when appending source files/directories, make the
original root become a subdirectory in the new root
called <name>, rather than adding the new source items
to the original root
Mksquashfs runtime options:
-version print version, licence and copyright message
-exit-on-error treat normally ignored errors as fatal
-recover <name> recover filesystem data using recovery file <name>
-no-recovery don't generate a recovery file
-recovery-path <name> use <name> as the directory to store the recovery file
-quiet no verbose output
-info print files written to filesystem
-no-progress don't display the progress bar
-progress display progress bar when using the -info option
-throttle <percentage> throttle the I/O input rate by the given percentage.
This can be used to reduce the I/O and CPU consumption
of Mksquashfs
-limit <percentage> limit the I/O input rate to the given percentage.
This can be used to reduce the I/O and CPU consumption
of Mksquashfs (alternative to -throttle)
-processors <number> Use <number> processors. By default will use number of
processors available
-mem <size> Use <size> physical memory. Currently set to 4096M
Optionally a suffix of K, M or G can be given to specify
Kbytes, Mbytes or Gbytes respectively
Miscellaneous options:
-root-owned alternative name for -all-root
-offset <offset> Skip <offset> bytes at the beginning of <dest>.
Optionally a suffix of K, M or G can be given to specify
Kbytes, Mbytes or Gbytes respectively.
Default 0 bytes.
-o <offset> synonym for -offset
-noInodeCompression alternative name for -noI
-noIdTableCompression alternative name for -noId
-noDataCompression alternative name for -noD
-noFragmentCompression alternative name for -noF
-noXattrCompression alternative name for -noX
-help output this options text to stdout
-h output this options text to stdout
-Xhelp print compressor options for selected compressor
Compressors available and compressor specific options:
gzip (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 9)
-Xwindow-size <window-size>
<window-size> should be 8 .. 15 (default 15)
-Xstrategy strategy1,strategy2,...,strategyN
Compress using strategy1,strategy2,...,strategyN in turn
and choose the best compression.
Available strategies: default, filtered, huffman_only,
run_length_encoded and fixed
lzo
-Xalgorithm <algorithm>
Where <algorithm> is one of:
lzo1x_1
lzo1x_1_11
lzo1x_1_12
lzo1x_1_15
lzo1x_999 (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 8)
Only applies to lzo1x_999 algorithm
lz4
-Xhc
Compress using LZ4 High Compression
xz
-Xbcj filter1,filter2,...,filterN
Compress using filter1,filter2,...,filterN in turn
(in addition to no filter), and choose the best compression.
Available filters: x86, arm, armthumb, powerpc, sparc, ia64
-Xdict-size <dict-size>
Use <dict-size> as the XZ dictionary size. The dictionary size
can be specified as a percentage of the block size, or as an
absolute value. The dictionary size must be less than or equal
to the block size and 8192 bytes or larger. It must also be
storable in the xz header as either 2^n or as 2^n+2^(n+1).
Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K
etc.
zstd
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 22 (default 15)
Source1 source2 ... are the source directories/files containing the
files/directories that will form the squashfs filesystem. If a single
directory is specified (i.e. mksquashfs source output_fs) the squashfs
filesystem will consist of that directory, with the top-level root
directory corresponding to the source directory.
If multiple source directories or files are specified, mksquashfs will merge
the specified sources into a single filesystem, with the root directory
containing each of the source files/directories. The name of each directory
entry will be the basename of the source path. If more than one source
entry maps to the same name, the conflicts are named xxx_1, xxx_2, etc. where
xxx is the original name.
To make this clear, take two example directories. Source directory
"/home/phillip/test" contains "file1", "file2" and "dir1".
Source directory "goodies" contains "goodies1", "goodies2" and "goodies3".
usage example 1:
%mksquashfs /home/phillip/test output_fs
This will generate a squashfs filesystem with root entries
"file1", "file2" and "dir1".
example 2:
%mksquashfs /home/phillip/test goodies output_fs
This will create a squashfs filesystem with the root containing
entries "test" and "goodies" corresponding to the source
directories "/home/phillip/test" and "goodies".
example 3:
%mksquashfs /home/phillip/test goodies test output_fs
This is the same as the previous example, except a third
source directory "test" has been specified. This conflicts
with the first directory named "test" and will be renamed "test_1".
Multiple sources allow filesystems to be generated without needing to
copy all source files into a common directory. This simplifies creating
filesystems.
The -keep-as-directory option can be used when only one source directory
is specified, and you wish the root to contain that directory, rather than
the contents of the directory. For example:
example 4:
%mksquashfs /home/phillip/test output_fs -keep-as-directory
This is the same as example 1, except for -keep-as-directory.
This will generate a root directory containing directory "test",
rather than the "test" directory contents "file1", "file2" and "dir1".
The Dest argument is the destination where the squashfs filesystem will be
written. This can either be a conventional file or a block device. If the file
doesn't exist it will be created, if it does exist and a squashfs
filesystem exists on it, mksquashfs will append. The -noappend option will
write a new filesystem irrespective of whether an existing filesystem is
present.
3.2 Changing compression algorithm and compression specific options
-------------------------------------------------------------------
By default Mksquashfs will compress using the gzip compression
algorithm. This algorithm offers a good trade-off between compression
ratio, and memory and time taken to decompress.
Squashfs also supports LZ4, LZO and XZ (LZMA2) compression. LZO offers worse
compression ratio than gzip, but is faster to decompress. XZ offers better
compression ratio than gzip, but at the expense of greater memory and time
to decompress (and significantly more time to compress). LZ4 is similar
to LZO, but, support for it is not yet in the mainline kernel, and so
its usefulness is currently limited to using Squashfs with Mksquashfs/Unsquashfs
as an archival system like tar.
If you're not building the squashfs-tools and kernel from source, then
the tools and kernel may or may not have been built with support for LZ4, LZO or
XZ compression. The compression algorithms supported by the build of
Mksquashfs can be found by typing mksquashfs without any arguments. The
compressors available are displayed at the end of the help message, e.g.
Compressors available and compressor specific options:
gzip (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 9)
-Xwindow-size <window-size>
<window-size> should be 8 .. 15 (default 15)
-Xstrategy strategy1,strategy2,...,strategyN
Compress using strategy1,strategy2,...,strategyN in turn
and choose the best compression.
Available strategies: default, filtered, huffman_only,
run_length_encoded and fixed
lzo
-Xalgorithm <algorithm>
Where <algorithm> is one of:
lzo1x_1
lzo1x_1_11
lzo1x_1_12
lzo1x_1_15
lzo1x_999 (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 8)
Only applies to lzo1x_999 algorithm
lz4
-Xhc
Compress using LZ4 High Compression
xz
-Xbcj filter1,filter2,...,filterN
Compress using filter1,filter2,...,filterN in turn
(in addition to no filter), and choose the best compression.
Available filters: x86, arm, armthumb, powerpc, sparc, ia64
-Xdict-size <dict-size>
Use <dict-size> as the XZ dictionary size. The dictionary size
can be specified as a percentage of the block size, or as an
absolute value. The dictionary size must be less than or equal
to the block size and 8192 bytes or larger. It must also be
storable in the xz header as either 2^n or as 2^n+2^(n+1).
Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K
etc.
zstd
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 22 (default 15)
If the compressor offers compression specific options (all the compressors now
have compression specific options except the deprecated lzma1 compressor)
then these options are also displayed (.i.e. in the above XZ is shown with two
compression specific options). The compression specific options are, obviously,
specific to the compressor in question, and the compressor documentation and
web sites should be consulted to understand their behaviour. In general
the Mksquashfs compression defaults for each compressor are optimised to
give the best performance for each compressor, where what constitutes
best depends on the compressor. For gzip/xz best means highest compression,
for LZO/LZ4 best means a tradeoff between compression and (de)-compression
overhead (LZO/LZ4 by definition are intended for weaker processors).
3.3 Changing global compression defaults used in mksquashfs
-----------------------------------------------------------
There are a large number of options that can be used to control the
compression in mksquashfs. By and large the defaults are the most
optimum settings and should only be changed in exceptional circumstances!
Note, this does not apply to the block size, increasing the block size
from the default of 128Kbytes will increase compression (especially
for the xz compressor) and should increase I/O performance too. However,
a block size of greater than 128Kbytes may increase latency in certain
cases (where the filesystem contains lots of fragments, and no locality
of reference is observed). For this reason the block size default is
configured to the less optimal 128Kbytes. Users should experiment
with 256Kbyte sizes or above.
The -noI, -noD and -noF options (also -noInodeCompression, -noDataCompression
and -noFragmentCompression) can be used to force mksquashfs to not compress
inodes/directories, data and fragments respectively. Giving all options
generates an uncompressed filesystem.
The -no-fragments tells mksquashfs to not generate fragment blocks, and rather
generate a filesystem similar to a Squashfs 1.x filesystem. It will of course
still be a Squashfs 4.0 filesystem but without fragments, and so it won't be
mountable on a Squashfs 1.x system.
The -always-use-fragments option tells mksquashfs to always generate
fragments for files irrespective of the file length. By default only small
files less than the block size are packed into fragment blocks. The ends of
files which do not fit fully into a block, are NOT by default packed into
fragments. To illustrate this, a 100K file has an initial 64K block and a 36K
remainder. This 36K remainder is not packed into a fragment by default. This
is because to do so leads to a 10 - 20% drop in sequential I/O performance, as a
disk head seek is needed to seek to the initial file data and another disk seek
is need to seek to the fragment block. Specify this option if you want file
remainders to be packed into fragment blocks. Doing so may increase the
compression obtained BUT at the expense of I/O speed.
The -no-duplicates option tells mksquashfs to not check the files being
added to the filesystem for duplicates. This can result in quicker filesystem
generation and appending although obviously compression will suffer badly if
there is a lot of duplicate files.
The -b option allows the block size to be selected, both "K" and "M" postfixes
are supported, this can be either 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K or
1M bytes.
3.4 Specifying the UIDs/GIDs used in the filesystem
---------------------------------------------------
By default files in the generated filesystem inherit the UID and GID ownership
of the original file. However, mksquashfs provides a number of options which
can be used to override the ownership.
The options -all-root and -root-owned (both do exactly the same thing) force all
file uids/gids in the generated Squashfs filesystem to be root. This allows
root owned filesystems to be built without root access on the host machine.
The "-force-uid uid" option forces all files in the generated Squashfs
filesystem to be owned by the specified uid. The uid can be specified either by
name (i.e. "root") or by number.
The "-force-gid gid" option forces all files in the generated Squashfs
filesystem to be group owned by the specified gid. The gid can be specified
either by name (i.e. "root") or by number.
3.5 Excluding files from the filesystem
---------------------------------------
The -e and -ef options allow files/directories to be specified which are
excluded from the output filesystem. The -e option takes the exclude
files/directories from the command line, the -ef option takes the
exlude files/directories from the specified exclude file, one file/directory
per line.
Two styles of exclude file matching are supported: basic exclude matching, and
extended wildcard matching. Basic exclude matching is a legacy feature
retained for backwards compatibility with earlier versions of Mksquashfs.
Extended wildcard matching should be used in preference.
3.5.1 Basic exclude matching
----------------------------
Each exclude file is treated as an exact match of a file/directory in
the source directories. If an exclude file/directory is absolute (i.e.
prefixed with /, ../, or ./) the entry is treated as absolute, however, if an
exclude file/directory is relative, it is treated as being relative to each of
the sources in turn, i.e.
%mksquashfs /tmp/source1 source2 output_fs -e ex1 /tmp/source1/ex2 out/ex3
Will generate exclude files /tmp/source1/ex2, /tmp/source1/ex1, source2/ex1,
/tmp/source1/out/ex3 and source2/out/ex3.
3.5.2 Extended exclude file handling
------------------------------------
Extended exclude file matching treats each exclude file as a wildcard or
regex expression. To enable wildcard matching specify the -wildcards
option, and to enable regex matching specify the -regex option. In most
cases the -wildcards option should be used rather than -regex because wildcard
matching behaviour is significantly easier to understand!
In addition to wildcards/regex expressions, exclude files can be "anchored" or
"non-anchored". An anchored exclude is one which matches from the root of the
directory and nowhere else, a non-anchored exclude matches anywhere. For
example given the directory hierarchy "a/b/c/a/b", the anchored exclude
"a/b" will match "a/b" at the root of the directory hierarchy, but
it will not match the "/a/b" sub-directory within directory "c", whereas a
non-anchored exclude would.
A couple of examples should make this clearer.
Anchored excludes
1. mksquashfs example image.sqsh -wildcards -e 'test/*.gz'
Exclude all files matching "*.gz" in the top level directory "test".
2. mksquashfs example image.sqsh -wildcards -e '*/[Tt]est/example*'
Exclude all files beginning with "example" inside directories called
"Test" or "test", that occur inside any top level directory.
Using extended wildcards, negative matching is also possible.
3. mksquashfs example image.sqsh -wildcards -e 'test/!(*data*).gz'
Exclude all files matching "*.gz" in top level directory "test",
except those with "data" in the name.
Non-anchored excludes
By default excludes match from the top level directory, but it is
often useful to exclude a file matching anywhere in the source directories.
For this non-anchored excludes can be used, specified by pre-fixing the
exclude with "...".
Examples:
1. mksquashfs example image.sqsh -wildcards -e '... *.gz'
Exclude files matching "*.gz" anywhere in the source directories.
For example this will match "example.gz", "test/example.gz", and
"test/test/example.gz".
2. mksquashfs example image.sqsh -wildcards -e '... [Tt]est/*.gz'
Exclude files matching "*.gz" inside directories called "Test" or
"test" that occur anywhere in the source directories.
Again, using extended wildcards, negative matching is also possible.
3. mksquashfs example image.sqsh -wildcards -e '... !(*data*).gz'
Exclude all files matching "*.gz" anywhere in the source directories,
except those with "data" in the name.
3.5.3 Exclude files summary
---------------------------
The -e and -ef exclude options are usefully used in archiving the entire
filesystem, where it is wished to avoid archiving /proc, and the filesystem
being generated, i.e.
%mksquashfs / /tmp/root.sqsh -e proc /tmp/root.sqsh
Multiple -ef options can be specified on the command line, and the -ef
option can be used in conjuction with the -e option.
3.6 Appending to squashfs filesystems
-------------------------------------
Running squashfs with the destination directory containing an existing
filesystem will add the source items to the existing filesystem. By default,
the source items are added to the existing root directory.
To make this clear... An existing filesystem "image" contains root entries
"old1", and "old2". Source directory "/home/phillip/test" contains "file1",
"file2" and "dir1".
example 1:
%mksquashfs /home/phillip/test image
Will create a new "image" with root entries "old1", "old2", "file1", "file2" and
"dir1"
example 2:
%mksquashfs /home/phillip/test image -keep-as-directory
Will create a new "image" with root entries "old1", "old2", and "test".
As shown in the previous section, for single source directories
'-keep-as-directory' adds the source directory rather than the
contents of the directory.
example 3:
%mksquashfs /home/phillip/test image -keep-as-directory -root-becomes
original-root
Will create a new "image" with root entries "original-root", and "test". The
'-root-becomes' option specifies that the original root becomes a subdirectory
in the new root, with the specified name.
The append option with file duplicate detection, means squashfs can be
used as a simple versioning archiving filesystem. A squashfs filesystem can
be created with for example the linux-2.4.19 source. Appending the linux-2.4.20
source will create a filesystem with the two source trees, but only the
changed files will take extra room, the unchanged files will be detected as
duplicates.
3.7 Appending recovery file feature
-----------------------------------
Recovery files are created when appending to existing Squashfs
filesystems. This allows the original filesystem to be recovered
if Mksquashfs aborts unexpectedly (i.e. power failure).
The recovery files are called squashfs_recovery_xxx_yyy, where
"xxx" is the name of the filesystem being appended to, and "yyy" is a
number to guarantee filename uniqueness (the PID of the parent Mksquashfs
process).
Normally if Mksquashfs exits correctly the recovery file is deleted to
avoid cluttering the filesystem. If Mksquashfs aborts, the "-recover"
option can be used to recover the filesystem, giving the previously
created recovery file as a parameter, i.e.
mksquashfs dummy image.sqsh -recover squashfs_recovery_image.sqsh_1234
The writing of the recovery file can be disabled by specifying the
"-no-recovery" option.
3.8 Pseudo file support
-----------------------
Mksquashfs supports pseudo files, these allow fake files, directories, character
and block devices to be specified and added to the Squashfs filesystem being
built, rather than requiring them to be present in the source directories.
This, for example, allows device nodes to be added to the filesystem without
requiring root access.
Mksquashfs 4.1 added support for "dynamic pseudo files" and a modify operation.
Dynamic pseudo files allow files to be dynamically created when Mksquashfs
is run, their contents being the result of running a command or piece of
shell script. The modifiy operation allows the mode/uid/gid of an existing
file in the source filesystem to be modified.
Mksquashfs 4.4 adds support for Symbolic links.
Two Mksquashfs options are supported, -p allows one pseudo file to be specified
on the command line, and -pf allows a pseudo file to be specified containing a
list of pseduo definitions, one per line.
3.8.1. Creating a dynamic file
------------------------------
Pseudo definition
Filename f mode uid gid command
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
command can be an executable or a piece of shell script, and it is executed
by running "/bin/sh -c command". The stdout becomes the contents of
"Filename".
Examples:
Running a basic command
-----------------------
/somedir/dmesg f 444 root root dmesg
creates a file "/somedir/dmesg" containing the output from dmesg.
Executing shell script
----------------------
RELEASE f 444 root root \
if [ ! -e /tmp/ver ]; then \
echo 0 > /tmp/ver; \
fi; \
ver=`cat /tmp/ver`; \
ver=$((ver +1)); \
echo $ver > /tmp/ver; \
echo -n `cat /tmp/release`; \
echo "-dev #"$ver `date` "Build host" `hostname`
Creates a file RELEASE containing the release name, date, build host, and
an incrementing version number. The incrementing version is a side-effect
of executing the shell script, and ensures every time Mksquashfs is run a
new version number is used without requiring any other shell scripting.
The above example also shows that commands can be split across multiple lines
using "\". Obviously as the script will be presented to the shell as a single
line, a semicolon is need to separate individual shell commands within the
shell script.
Reading from a device (or fifo/named socket)
--------------------------------------------
input f 444 root root dd if=/dev/sda1 bs=1024 count=10
Copies 10K from the device /dev/sda1 into the file input. Ordinarily Mksquashfs
given a device, fifo, or named socket will place that special file within the
Squashfs filesystem, the above allows input from these special files to be
captured and placed in the Squashfs filesystem.
3.8.2. Creating a block or character device
-------------------------------------------
Pseudo definition
Filename type mode uid gid major minor
Where type is either
b - for block devices, and
c - for character devices
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
/dev/chr_dev c 666 root root 100 1
/dev/blk_dev b 666 0 0 200 200
creates a character device "/dev/chr_dev" with major:minor 100:1 and
a block device "/dev/blk_dev" with major:minor 200:200, both with root
uid/gid and a mode of rw-rw-rw.
3.8.3. Creating a directory
---------------------------
Pseudo definition
Filename d mode uid gid
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
/pseudo_dir d 666 root root
creates a directory "/pseudo_dir" with root uid/gid and mode of rw-rw-rw.
3.8.4. Creating a symbolic link
-------------------------------
Pseudo definition
Filename s mode uid gid symlink
uid and gid can be either specified as a decimal number, or by name.
Note mode is ignored, as symlinks always have "rwxrwxrwx" permissions.
For example:
symlink s 0 root root example
creates a symlink "symlink" to file "example" with root uid/gid.
3.8.5. Modifying attributes of an existing file
-----------------------------------------------
Pseudo definition
Filename m mode uid gid
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
dmesg m 666 root root
Changes the attributes of the file "dmesg" in the filesystem to have
root uid/gid and a mode of rw-rw-rw, overriding the attributes obtained
from the source filesystem.
3.9 Miscellaneous options
-------------------------
The -info option displays the files/directories as they are compressed and
added to the filesystem. The original uncompressed size of each file
is printed, along with DUPLICATE if the file is a duplicate of a
file in the filesystem.
The -nopad option informs mksquashfs to not pad the filesystem to a 4K multiple.
This is performed by default to enable the output filesystem file to be mounted
by loopback, which requires files to be a 4K multiple. If the filesystem is
being written to a block device, or is to be stored in a bootimage, the extra
pad bytes are not needed.
4. UNSQUASHFS
-------------
Unsquashfs allows you to decompress and extract a Squashfs filesystem without
mounting it. It can extract the entire filesystem, or a specific
file or directory.
Unsquashfs can decompress all official Squashfs filesystem versions.
The Unsquashfs usage info is:
SYNTAX: unsquashfs [options] filesystem [files to extract or exclude (with -excludes) or cat (with -cat )]
-v[ersion] print version, licence and copyright information
-cat cat the listed files to stdout
-d[est] <pathname> unsquash to <pathname>, default "squashfs-root"
-max[-depth] <levels> descend at most <levels> of directories when
unsquashing or listing
-excludes treat files on command line as exclude files
-ex[clude-list] list of files to be excluded, terminated with ;
-follow[-symlinks] follow symlinks in extract files, and add all
files/symlinks needed to resolve extract file.
Implies -no-wildcards
-missing[-symlinks] Unsquashfs will abort if any symlink can't be
resolved in -follow-symlinks
-q[uiet] no verbose output
-n[o-progress] don't display the progress bar
-no[-xattrs] don't extract xattrs in file system
-x[attrs] extract xattrs in file system (default)
-u[ser-xattrs] only extract user xattrs in file system.
Enables extracting xattrs
-p[rocessors] <number> use <number> processors. By default will use
number of processors available
-i[nfo] print files as they are unsquashed
-li[nfo] print files as they are unsquashed with file
attributes (like ls -l output)
-l[s] list filesystem, but don't unsquash
-ll[s] list filesystem with file attributes (like
ls -l output), but don't unsquash
-lln[umeric] -lls but with numeric uids and gids
-lc list filesystem concisely, displaying only files
and empty directories. Don't unsquash
-llc list filesystem concisely with file attributes,
displaying only files and empty directories.
Don't unsquash
-o[ffset] <bytes> skip <bytes> at start of <dest>. Optionally a
suffix of K, M or G can be given to specify
Kbytes, Mbytes or Gbytes respectively (default
0 bytes).
-f[orce] if file already exists then overwrite
-ig[nore-errors] treat errors writing files to output as
non-fatal
-st[rict-errors] treat all errors as fatal
-no-exit[-code] don't set exit code (to nonzero) on non-fatal
errors
-s[tat] display filesystem superblock information
-UTC use UTC rather than local time zone when
displaying time
-mkfs-time display filesystem superblock time
-fstime synonym for -mkfs-time
-extract-file <file> list of directories or files to extract.
One per line
-exclude-file <file> list of directories or files to exclude.
One per line
-pf <file> output a pseudo file equivalent of the input
Squashfs filesystem
-pseudo-file <file> alternative name for -pf
-e[f] <extract file> synonym for -extract-file
-exc[f] <exclude file> synonym for -exclude-file
-da[ta-queue] <size> set data queue to <size> Mbytes. Default 256
Mbytes
-fr[ag-queue] <size> set fragment queue to <size> Mbytes. Default
256 Mbytes
-no-wild[cards] do not use wildcard matching in extract names
-r[egex] treat extract names as POSIX regular expressions
rather than use the default shell wildcard
expansion (globbing)
-L synonym for -follow-symlinks
-h[elp] output this options text to stdout