Skip to content

Commit

Permalink
MSYS2 target support (cossacklabs#469)
Browse files Browse the repository at this point in the history
* MSYS2 target support

MSYS2 is a popular and easy way to compile POSIX-compliant software on
Windows. It provides GNU development environment and API compatibility
layer between POSIX and WinAPI. All of this allows us to use the same
build system and code as is, with the toolchain automagically producing
PE binaries useable on Windows.

However, we need to introduce a number of changes to the build system
to support MSYS:

- Detect MSYS environment and provide "IS_MSYS" variable for checks

  This allows us to introduce MSYS-specific behavior in Makefile.

- Build Windows DLLs for shared objects

  First of all, use MSYS naming scheme for dynamically loaded libraries:
  "msys-${library_name}-${abi_version}.dll". These will contain the code
  loaded by applications.

  After that we need to produce "import libraries" which are necessary
  in Windows world to link against dynamic libraries. On MSYS they are
  named "lib${library_name}.dll.a".

  Regular static libraries are still named "lib${library_name}.a". These
  are useable only for inside MSYS2.

- Install libraries to correct paths

  On MSYS shared libraries have to be installed to "/usr/bin" because of
  the way library loading works in Windows: libraries need to be placed
  next to executables.

  DLLs do not need symlinks on MSYS. However, they need import libraries
  instead. Update the installation targets to do this correctly.

  Update uninstallation targets as well.

- Avoid duplicate slashes when usign DESTDIR

  Remove slashes between DESTDIR and paths. Windows does not normalize
  duplicate slashes under MSYS which leads to really weird issues when
  creating directories.

- Place tests into the same directory as libraries

  We need to do this because tests link against libraries dynamically,
  they need to be able to load the library from the build directory
  (as it's not yet installed). Windows does not have a direct analog
  of RPATH (as on Linux), and does not record library paths (as macOS
  does). We can use LD_LIBRARY_PATH for running tests, but that would
  make them hard to use without our makefile. Therefore just change
  the location.

Also, the NIST test suite has a bit different name on Windows. Add it to
the .gitignore list.

With this, installation on MSYS can be done like this:

    make install

GGEZ!

(Note that MSYS does not have /usr/local in its default search paths.
You'll need to use -L/usr/local/lib when building applications, and
add /usr/local/bin to PATH when running them. Or just install with
PREFIX=/usr)

* Packaging for MSYS2 (and pacman)

Provide a PKGBUILD file which can be used to build Themis package for
MSYS2. A similar file could be used for building on ArchLinux. However,
this one does not work as is on Linux due to differences in dependency
naming and directory layout.

Themis is a library. It is customary in MSYS2 to provide Debian-style
split packages for libraries:

  - "themis" contains runtime shared libraries: *.dll on Windows
  - "themis-devel" contains headers, static libraries, and pkg-config
    files which are required for building software that uses Themis

The packages may be built by running

    makepkg -p PKGBUILD.MSYS2

in the directory with the PKGBUILD file, then you install them with

    pacman -U themis-0.11.2-1-x86_64.tar.xz
    pacman -U themis-devel-0.11.2-1-x86_64.tar.xz

makepkg always downloads pristine source tarball, so there is no point
in adding a Makefile target for preparing a package from source code.
  • Loading branch information
ilammy authored May 22, 2019
1 parent 5ea9e31 commit 7cb03be
Show file tree
Hide file tree
Showing 7 changed files with 157 additions and 39 deletions.
11 changes: 9 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,14 @@ BUILD_CMD_ = LOG=$$($(CMD) 2>&1) ; if [ $$? -ne 0 ]; then $(PRINT_ERROR_); elif

PKGINFO_PATH = PKGINFO

UNAME=$(shell uname)
UNAME := $(shell uname)

ifeq ($(UNAME),Darwin)
IS_MACOS := true
else ifeq ($(UNAME),Linux)
IS_LINUX := true
else ifeq ($(shell uname -o),Msys)
IS_MSYS := true
endif

ifneq ($(shell $(CC) --version 2>&1 | grep -oi "Emscripten"),)
Expand Down Expand Up @@ -111,6 +113,7 @@ endif
# default installation paths
prefix = $(PREFIX)
exec_prefix = $(prefix)
bindir = $(prefix)/bin
includedir = $(prefix)/include
libdir = $(exec_prefix)/lib
pkgconfigdir = $(libdir)/pkgconfig
Expand Down Expand Up @@ -208,7 +211,7 @@ endif

SHARED_EXT = so

ifeq ($(shell uname),Darwin)
ifdef IS_MACOS
SHARED_EXT = dylib
ifneq ($(SDK),)
SDK_PLATFORM_VERSION=$(shell xcrun --sdk $(SDK) --show-sdk-platform-version)
Expand All @@ -225,6 +228,10 @@ CFLAFS += -arch $(ARCH)
endif
endif

ifdef IS_MSYS
SHARED_EXT = dll
endif

ifneq ($(shell $(CC) --version 2>&1 | grep -E -i -c "clang version"),0)
IS_CLANG_COMPILER := true
endif
Expand Down
64 changes: 64 additions & 0 deletions PKGBUILD.MSYS2
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Themis build script for pacman in MSYS2 environment
#
# Maintainer: Cossack Labs Limited <dev@cossacklabs.com>

pkgname=('themis' 'themis-devel')
pkgbase=themis
pkgver=0.11.2
pkgrel=1

pkgdesc="Data security library for network communication and data storage"
url="https://www.cossacklabs.com/themis/"
license=('Apache')
changelog=CHANGELOG.md

arch=('x86_64' 'i686')
depends=('libopenssl>=1.1.1')
makedepends=('tar' 'gcc' 'make' 'openssl-devel>=1.1.1')

source=("https://github.com/cossacklabs/themis/archive/$pkgver.tar.gz")
sha256sums=('ee5c4be360401094dc5429035e1499912b3ccc10e93f71d0c2578934a5b2c81f')
sha1sums=('418d71b0d3dd1ffff93ef025908bacb99d4c4aa4')
md5sums=('afc3ac4e7d6c6db59e4e2f059a4fc973')
# TODO: verify package signature

# Unfortunately, bsdtar cannot handle symlinks on MSYS2 [1] so we have to use
# regular tar for source code extraction, and make sure that files are removed
# before extracting them.
#
# [1]: https://github.com/msys2/MSYS2-packages/issues/140
noextract=("$pkgver.tar.gz")
prepare() {
tar --unlink-first --recursive-unlink -xf "$pkgver.tar.gz"
}

build() {
cd "$pkgbase-$pkgver"
make PREFIX=/usr
}

check() {
cd "$pkgname-$pkgver"
# Skip NIST STS tests which take long time and are prone to spurious
# failures in virtualized environments.
make -k test NO_NIST_STS=1
}

# Themis does not provide separate installation targets. It's easier to just
# install everything and then remove unnecessary files for package splitting.

package_themis() {
cd "$pkgbase-$pkgver"
make install PREFIX=/usr DESTDIR="$pkgdir/"
rm -rf "$pkgdir/usr/include"
rm -rf "$pkgdir/usr/lib"
}

package_themis-devel() {
pkgdesc="Themis header files and development libraries"
depends=("themis=$pkgver")

cd "$pkgbase-$pkgver"
make install PREFIX=/usr DESTDIR="$pkgdir/"
rm -rf "$pkgdir/usr/bin"
}
53 changes: 37 additions & 16 deletions src/soter/soter.mk
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ endif
ifdef IS_MACOS
LIBSOTER_SO = libsoter.$(LIBRARY_SO_VERSION).$(SHARED_EXT)
endif
ifdef IS_MSYS
LIBSOTER_SO = msys-soter-$(LIBRARY_SO_VERSION).$(SHARED_EXT)
LIBSOTER_LINK =
LIBSOTER_IMPORT = libsoter.dll.a
LIBSOTER_SO_LDFLAGS = -Wl,-out-implib,$(BIN_PATH)/$(LIBSOTER_IMPORT)
endif

SOTER_SOURCES = $(wildcard $(SRC_PATH)/soter/*.c)
SOTER_HEADERS = $(wildcard $(SRC_PATH)/soter/*.h)
Expand Down Expand Up @@ -79,27 +85,42 @@ $(BIN_PATH)/libsoter.pc:

install_soter: err $(BIN_PATH)/$(LIBSOTER_A) $(BIN_PATH)/$(LIBSOTER_SO) $(BIN_PATH)/libsoter.pc
@echo -n "install Soter "
@mkdir -p $(DESTDIR)/$(includedir)/soter
@mkdir -p $(DESTDIR)/$(pkgconfigdir)
@mkdir -p $(DESTDIR)/$(libdir)
@$(INSTALL_DATA) $(SRC_PATH)/soter/*.h $(DESTDIR)/$(includedir)/soter
@$(INSTALL_DATA) $(BIN_PATH)/libsoter.pc $(DESTDIR)/$(pkgconfigdir)
@$(INSTALL_DATA) $(BIN_PATH)/$(LIBSOTER_A) $(DESTDIR)/$(libdir)
@$(INSTALL_PROGRAM) $(BIN_PATH)/$(LIBSOTER_SO) $(DESTDIR)/$(libdir)
@mkdir -p $(DESTDIR)$(includedir)/soter
@mkdir -p $(DESTDIR)$(pkgconfigdir)
ifdef IS_MSYS
@mkdir -p $(DESTDIR)$(bindir)
endif
@mkdir -p $(DESTDIR)$(libdir)
@$(INSTALL_DATA) $(SRC_PATH)/soter/*.h $(DESTDIR)$(includedir)/soter
@$(INSTALL_DATA) $(BIN_PATH)/libsoter.pc $(DESTDIR)$(pkgconfigdir)
@$(INSTALL_DATA) $(BIN_PATH)/$(LIBSOTER_A) $(DESTDIR)$(libdir)
ifdef IS_MSYS
@$(INSTALL_PROGRAM) $(BIN_PATH)/$(LIBSOTER_SO) $(DESTDIR)$(bindir)
else
@$(INSTALL_PROGRAM) $(BIN_PATH)/$(LIBSOTER_SO) $(DESTDIR)$(libdir)
endif
ifdef IS_MACOS
@install_name_tool -id "$(libdir)/$(LIBSOTER_SO)" "$(DESTDIR)/$(libdir)/$(LIBSOTER_SO)"
@install_name_tool -change "$(BIN_PATH)/$(LIBSOTER_SO)" "$(libdir)/$(LIBSOTER_SO)" "$(DESTDIR)/$(libdir)/$(LIBSOTER_SO)"
@install_name_tool -id "$(libdir)/$(LIBSOTER_SO)" "$(DESTDIR)$(libdir)/$(LIBSOTER_SO)"
@install_name_tool -change "$(BIN_PATH)/$(LIBSOTER_SO)" "$(libdir)/$(LIBSOTER_SO)" "$(DESTDIR)$(libdir)/$(LIBSOTER_SO)"
endif
ifneq ($(LIBSOTER_SO),$(LIBSOTER_LINK))
@ln -sf $(LIBSOTER_SO) $(DESTDIR)/$(libdir)/$(LIBSOTER_LINK)
ifneq ($(LIBSOTER_IMPORT),)
@$(INSTALL_DATA) $(BIN_PATH)/$(LIBSOTER_IMPORT) $(DESTDIR)$(libdir)
endif
ifneq ($(LIBSOTER_LINK),)
@ln -sf $(LIBSOTER_SO) $(DESTDIR)$(libdir)/$(LIBSOTER_LINK)
endif
@$(PRINT_OK_)

uninstall_soter:
@echo -n "uninstall Soter "
@rm -rf $(DESTDIR)/$(includedir)/soter
@rm -f $(DESTDIR)/$(pkgconfigdir)/libsoter.pc
@rm -f $(DESTDIR)/$(libdir)/$(LIBSOTER_A)
@rm -f $(DESTDIR)/$(libdir)/$(LIBSOTER_SO)
@rm -f $(DESTDIR)/$(libdir)/$(LIBSOTER_LINK)
@rm -rf $(DESTDIR)$(includedir)/soter
@rm -f $(DESTDIR)$(pkgconfigdir)/libsoter.pc
@rm -f $(DESTDIR)$(libdir)/$(LIBSOTER_A)
ifdef IS_MSYS
@rm -f $(DESTDIR)$(bindir)/$(LIBSOTER_SO)
@rm -f $(DESTDIR)$(libdir)/$(LIBSOTER_IMPORT)
else
@rm -f $(DESTDIR)$(libdir)/$(LIBSOTER_SO)
@rm -f $(DESTDIR)$(libdir)/$(LIBSOTER_LINK)
endif
@$(PRINT_OK_)
55 changes: 38 additions & 17 deletions src/themis/themis.mk
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ endif
ifdef IS_MACOS
LIBTHEMIS_SO = libthemis.$(LIBRARY_SO_VERSION).$(SHARED_EXT)
endif
ifdef IS_MSYS
LIBTHEMIS_SO = msys-themis-$(LIBRARY_SO_VERSION).$(SHARED_EXT)
LIBTHEMIS_LINK =
LIBTHEMIS_IMPORT = libthemis.dll.a
LIBTHEMIS_SO_LDFLAGS = -Wl,-out-implib,$(BIN_PATH)/$(LIBTHEMIS_IMPORT)
endif

THEMIS_SOURCES = $(wildcard $(SRC_PATH)/themis/*.c)
THEMIS_HEADERS = $(wildcard $(SRC_PATH)/themis/*.h)
Expand Down Expand Up @@ -69,28 +75,43 @@ $(BIN_PATH)/libthemis.pc:

install_themis: err $(BIN_PATH)/$(LIBTHEMIS_A) $(BIN_PATH)/$(LIBTHEMIS_SO) $(BIN_PATH)/libthemis.pc
@echo -n "install Themis "
@mkdir -p $(DESTDIR)/$(includedir)/themis
@mkdir -p $(DESTDIR)/$(pkgconfigdir)
@mkdir -p $(DESTDIR)/$(libdir)
@$(INSTALL_DATA) $(SRC_PATH)/themis/*.h $(DESTDIR)/$(includedir)/themis
@$(INSTALL_DATA) $(BIN_PATH)/libthemis.pc $(DESTDIR)/$(pkgconfigdir)
@$(INSTALL_DATA) $(BIN_PATH)/$(LIBTHEMIS_A) $(DESTDIR)/$(libdir)
@$(INSTALL_PROGRAM) $(BIN_PATH)/$(LIBTHEMIS_SO) $(DESTDIR)/$(libdir)
@mkdir -p $(DESTDIR)$(includedir)/themis
@mkdir -p $(DESTDIR)$(pkgconfigdir)
ifdef IS_MSYS
@mkdir -p $(DESTDIR)$(bindir)
endif
@mkdir -p $(DESTDIR)$(libdir)
@$(INSTALL_DATA) $(SRC_PATH)/themis/*.h $(DESTDIR)$(includedir)/themis
@$(INSTALL_DATA) $(BIN_PATH)/libthemis.pc $(DESTDIR)$(pkgconfigdir)
@$(INSTALL_DATA) $(BIN_PATH)/$(LIBTHEMIS_A) $(DESTDIR)$(libdir)
ifdef IS_MSYS
@$(INSTALL_PROGRAM) $(BIN_PATH)/$(LIBTHEMIS_SO) $(DESTDIR)$(bindir)
else
@$(INSTALL_PROGRAM) $(BIN_PATH)/$(LIBTHEMIS_SO) $(DESTDIR)$(libdir)
endif
ifdef IS_MACOS
@install_name_tool -id "$(libdir)/$(LIBTHEMIS_SO)" "$(DESTDIR)/$(libdir)/$(LIBTHEMIS_SO)"
@install_name_tool -change "$(BIN_PATH)/$(LIBTHEMIS_SO)" "$(libdir)/$(LIBTHEMIS_SO)" "$(DESTDIR)/$(libdir)/$(LIBTHEMIS_SO)"
@install_name_tool -change "$(BIN_PATH)/$(LIBSOTER_SO)" "$(libdir)/$(LIBSOTER_SO)" "$(DESTDIR)/$(libdir)/$(LIBTHEMIS_SO)"
@install_name_tool -id "$(libdir)/$(LIBTHEMIS_SO)" "$(DESTDIR)$(libdir)/$(LIBTHEMIS_SO)"
@install_name_tool -change "$(BIN_PATH)/$(LIBTHEMIS_SO)" "$(libdir)/$(LIBTHEMIS_SO)" "$(DESTDIR)$(libdir)/$(LIBTHEMIS_SO)"
@install_name_tool -change "$(BIN_PATH)/$(LIBSOTER_SO)" "$(libdir)/$(LIBSOTER_SO)" "$(DESTDIR)$(libdir)/$(LIBTHEMIS_SO)"
endif
ifneq ($(LIBTHEMIS_SO),$(LIBTHEMIS_LINK))
@ln -sf $(LIBTHEMIS_SO) $(DESTDIR)/$(libdir)/$(LIBTHEMIS_LINK)
ifneq ($(LIBTHEMIS_IMPORT),)
@$(INSTALL_DATA) $(BIN_PATH)/$(LIBTHEMIS_IMPORT) $(DESTDIR)$(libdir)
endif
ifneq ($(LIBTHEMIS_LINK),)
@ln -sf $(LIBTHEMIS_SO) $(DESTDIR)$(libdir)/$(LIBTHEMIS_LINK)
endif
@$(PRINT_OK_)

uninstall_themis:
@echo -n "uninstall Themis "
@rm -rf $(DESTDIR)/$(includedir)/themis
@rm -f $(DESTDIR)/$(pkgconfigdir)/libthemis.pc
@rm -f $(DESTDIR)/$(libdir)/$(LIBTHEMIS_A)
@rm -f $(DESTDIR)/$(libdir)/$(LIBTHEMIS_SO)
@rm -f $(DESTDIR)/$(libdir)/$(LIBTHEMIS_LINK)
@rm -rf $(DESTDIR)$(includedir)/themis
@rm -f $(DESTDIR)$(pkgconfigdir)/libthemis.pc
@rm -f $(DESTDIR)$(libdir)/$(LIBTHEMIS_A)
ifdef IS_MSYS
@rm -f $(DESTDIR)$(bindir)/$(LIBTHEMIS_SO)
@rm -f $(DESTDIR)$(libdir)/$(LIBTHEMIS_IMPORT)
else
@rm -f $(DESTDIR)$(libdir)/$(LIBTHEMIS_SO)
@rm -f $(DESTDIR)$(libdir)/$(LIBTHEMIS_LINK)
endif
@$(PRINT_OK_)
1 change: 1 addition & 0 deletions tests/soter/nist-sts/.gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Generated files
assess
assess.exe
experiments/
obj/
rng.bin
6 changes: 4 additions & 2 deletions tests/soter/soter.mk
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,12 @@
# limitations under the License.
#

SOTER_TEST_BIN = $(TEST_BIN_PATH)/soter_test
ifdef IS_EMSCRIPTEN
SOTER_TEST_BIN = $(TEST_BIN_PATH)/soter_test.js
else
SOTER_TEST_BIN = $(TEST_BIN_PATH)/soter_test
endif
ifdef IS_MSYS
SOTER_TEST_BIN = $(BIN_PATH)/soter_test
endif

NIST_STS_DIR = tests/soter/nist-sts
Expand Down
6 changes: 4 additions & 2 deletions tests/themis/themis.mk
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,12 @@
# limitations under the License.
#

THEMIS_TEST_BIN = $(TEST_BIN_PATH)/themis_test
ifdef IS_EMSCRIPTEN
THEMIS_TEST_BIN = $(TEST_BIN_PATH)/themis_test.js
else
THEMIS_TEST_BIN = $(TEST_BIN_PATH)/themis_test
endif
ifdef IS_MSYS
THEMIS_TEST_BIN = $(BIN_PATH)/themis_test
endif

THEMIS_TEST_SOURCES = $(wildcard tests/themis/*.c)
Expand Down

0 comments on commit 7cb03be

Please sign in to comment.