Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

preload: init at 0.6.4 #141583

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions nixos/modules/module-list.nix
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@
./programs/partition-manager.nix
./programs/plotinus.nix
./programs/proxychains.nix
./programs/preload.nix
./programs/phosh.nix
./programs/qt5ct.nix
./programs/screen.nix
Expand Down
173 changes: 173 additions & 0 deletions nixos/modules/programs/preload.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
{
config,
lib,
pkgs,
...
}:
with lib; let
cfg = config.programs.preload;
settingsFormat = pkgs.formats.ini {};
configFile = settingsFormat.generate "preload.conf" cfg.settings;
in {
Comment on lines +1 to +11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{
config,
lib,
pkgs,
...
}:
with lib; let
cfg = config.programs.preload;
settingsFormat = pkgs.formats.ini {};
configFile = settingsFormat.generate "preload.conf" cfg.settings;
in {
{ config
, lib
, pkgs
, ...
}:
with lib;
let
cfg = config.programs.preload;
settingsFormat = pkgs.formats.ini {};
configFile = settingsFormat.generate "preload.conf" cfg.settings;
in {

options = {
programs.preload = {
enable = mkEnableOption "Preload loads files and executables to memory to speed up processes";
settings = mkOption {
type = types.submodule {
freeformType = settingsFormat.type;
};
apply = recursiveUpdate default;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default can't be referenced here. Just moving it to the let block works.

Also settings.default should be {} to allow just using the default config.


default = {
model = {
cycle = 20;
usecorrelation = "true";
minsize = 2000000;
memtotal = -10;
memfree = 50;
memcached = 0;
};
system = {
doscan = "true";
dopredict = "true";
autosave = 3600;
mapprefix = "/run/current-system/sw/share;/run/current-system/sw/lib;!/";
exeprefix = "!/run/current-system/sw/sbin/;/run/current-system/sw/bin;!/";
Comment on lines +34 to +35
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GustavoPeredo are you sure it works fine? the things that i am most concerned about are these 2

Copy link
Author

@GustavoPeredo GustavoPeredo Mar 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are following expected behavior, but it would be more useful if the nix store was added to these paths.

processes = 30;
sortstrategy = 3;
};
};
description = ''
Configuration for preload
'';
example = literalExpression ''
{
model = {
# This is the quantum of time for preload. Preload performs
# data gathering and predictions every cycle. Use an even
# number.
# Note: Setting this parameter too low may reduce system performance
# and stability.
# unit: seconds.
cycle = 20;
# Whether correlation coefficient should be used in the prediction
# algorithm. There are arguments both for and against using it.
# Currently it's believed that using it results in more accurate
# prediction. The option may be removed in the future.
usecorrelation = "true";
# Minimum sum of the length of maps of the process for
# preload to consider tracking the application.
# Note: Setting this parameter too high will make preload less
# effective, while setting it too low will make it eat
# quadratically more resources, as it tracks more processes.
# unit: bytes.
minsize = "2000000";
# Precentage of total memory
memtotal = "-10";
# Precentage of free memory
memfree = "50";
# Precentage of cached memory
memcached = "0";
};
system = {
# Whether preload should monitor running processes and update its
# model state. Normally you do want that, that's all preload is
# about, but you may want to temporarily turn it off for various
# reasons like testing and only make predictions. Note that if
# scanning is off, predictions are made based on whatever processes
# have been running when preload started and the list of running
# processes is not updated at all.
doscan = "true";
# Whether preload should make prediction and prefetch anything off
# the disk. Quite like doscan, you normally want that, that's the
# other half of what preload is about, but you may want to temporarily
# turn it off, to only train the model for example. Note that
# this allows you to turn scan/predict or or off on the fly, by
# modifying the config file and signalling the daemon.
dopredict = "true";
# Preload will automatically save the state to disk every
# autosave period. This is only relevant if doscan is set to true.
# Note that some janitory work on the model, like removing entries
# for files that no longer exist happen at state save time. So,
# turning off autosave completely is not advised.
# unit: seconds.
autosave = "3600";
# A list of path prefixes that control which mapped file are to
# be considered by preload and which not. The list items are
# separated by semicolons. Matching will be stopped as soon as
# the first item is matched. For each item, if item appears at
# the beginning of the path of the file, then a match occurs, and
# the file is accepted. If on the other hand, the item has a
# exclamation mark as its first character, then the rest of the
# item is considered, and if a match happens, the file is rejected.
# For example a value of !/lib/modules;/ means that every file other
# than those in /lib/modules should be accepted. In this case, the
# trailing item can be removed, since if no match occurs, the file is
# accepted. It's advised to make sure /dev is rejected, since
# preload doesn't special-handle device files internally.
# Note that /lib matches all of /lib, /lib64, and even /libexec if
# there was one. If one really meant /lib only, they should use
# /lib/ instead.
mapprefix = "/usr/;/lib;/var/cache/;!/";
# The syntax for this is exactly the same as for mapprefix. The only
# difference is that this is used to accept or reject binary exectuable
# files instead of maps.
exeprefix = "!/usr/sbin/;!/usr/local/sbin/;/usr/;!/";
# Maximum number of processes to use to do parallel readahead. If
# equal to 0, no parallel processing is done and all readahead is
# done in-process. Parallel readahead supposedly gives a better I/O
# performance as it allows the kernel to batch several I/O requests
# of nearby blocks.
processes = "30";
# The I/O sorting strategy. Ideally this should be automatically
# decided, but it's not currently. One of
# 0 -- SORT_NONE: No I/O sorting.
# Useful on Flash memory for example.
# 1 -- SORT_PATH: Sort based on file path only.
# Useful for network filesystems.
# 2 -- SORT_INODE: Sort based on inode number.
# Does less house-keeping I/O than the next option.
# 3 -- SORT_BLOCK: Sort I/O based on disk block. Most sophisticated.
# And useful for most Linux filesystems.
sortstrategy = "3";
};
};
'';
};
};
};

config = mkIf cfg.enable {
systemd = {
services.preload = {
description = "Adaptive readahead daemon";
wantedBy = ["multi-user.target"];
serviceConfig = {
Type = "simple";
ExecStart = "${pkgs.preload}/bin/preload --foreground --verbose 1 -c ${configFile}";
StateDirectory = "preload";
LogsDirectory = "preload";
IOSchedulingClass = 3;
};
};
};
};
}
55 changes: 55 additions & 0 deletions pkgs/misc/preload/default.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
lib,
stdenv,
fetchurl,
glib,
pkg-config,
help2man,
autoreconfHook,
}:
stdenv.mkDerivation rec {
Comment on lines +1 to +10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{
lib,
stdenv,
fetchurl,
glib,
pkg-config,
help2man,
autoreconfHook,
}:
stdenv.mkDerivation rec {
{ lib
, stdenv
, fetchurl
, glib
, pkg-config
, help2man
, autoreconfHook
}:
stdenv.mkDerivation rec {

pname = "preload";
version = "0.6.4";

src = fetchurl {
url = "mirror://sourceforge/${pname}/${pname}-${version}.tar.gz";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
url = "mirror://sourceforge/${pname}/${pname}-${version}.tar.gz";
url = "mirror://sourceforge/preload/preload-${version}.tar.gz";

sha256 = "d0a558e83cb29a51d9d96736ef39f4b4e55e43a589ad1aec594a048ca22f816b";
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
};
};
postPatch = ''
# "--logdir=/var/log/preload" failed with unknown option
substituteInPlace configure.ac \
--replace "logdir='\''${localstatedir}/log'" "logdir='\''${localstatedir}/log/preload'"
'';


nativeBuildInputs = [
pkg-config
autoreconfHook
help2man
];

buildInputs = [
glib
];

configureFlags = [
"--sysconfdir=/etc"
"--localstatedir=/var"
];

installFlags = [
"sbindir=$(out)/bin"
"sysconfdir=./discard" # configuration done using nixos module
"localstatedir=./discard" # discard the files and dirs, they're created in the module
];

postPatch = ''
# "--logdir=/var/log/preload" failed with unknown option
substituteInPlace configure.ac \
--replace "logdir='\''${localstatedir}/log'" "logdir='\''${localstatedir}/log/preload'"
'';
Comment on lines +39 to +44
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
postPatch = ''
# "--logdir=/var/log/preload" failed with unknown option
substituteInPlace configure.ac \
--replace "logdir='\''${localstatedir}/log'" "logdir='\''${localstatedir}/log/preload'"
'';

Copy link
Author

@GustavoPeredo GustavoPeredo Mar 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were these formatting changes tested? I want to commit them, but had no time to test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No but I am highly confident that they just work. Nix does not care about the ordering but it is very good practice to use somewhat standard ordering.


# loops '0kb available for preloading, using 0kb of it'
doCheck = false;

meta = with lib; {
homepage = "http://sourceforge.net/projects/preload";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
homepage = "http://sourceforge.net/projects/preload";
homepage = "https://sourceforge.net/projects/preload/";

description = "Makes applications run faster by prefetching binaries and shared objects";
license = licenses.gpl1;
platforms = platforms.linux;
};
}
2 changes: 2 additions & 0 deletions pkgs/top-level/all-packages.nix
Original file line number Diff line number Diff line change
Expand Up @@ -9074,6 +9074,8 @@ with pkgs;

pre-commit = callPackage ../tools/misc/pre-commit { };

preload = callPackage ../misc/preload {};

pretty-simple = callPackage ../development/tools/pretty-simple { };

prettyping = callPackage ../tools/networking/prettyping { };
Expand Down