Skip to content

Add AppImage support to Linux builds #3688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tresf merged 60 commits into from
Oct 16, 2017
Merged

Add AppImage support to Linux builds #3688

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
92eae8f
Initial version
tresf Jul 7, 2017
8dcbd19
Better messaging, rename and move to bulid dir
tresf Jul 7, 2017
5004eb7
Remove newlines
tresf Jul 7, 2017
74db001
Various fixes; untested
tresf Jul 8, 2017
91a000b
linuxdeployqt build inst, hack the .desktop file
tresf Jul 8, 2017
84de8cd
First working copy
tresf Jul 9, 2017
2b7dead
Fix download logic
tresf Jul 9, 2017
0e5ef88
Misc fixes
tresf Jul 9, 2017
62eea51
Remove unecessary command
tresf Jul 9, 2017
9beb316
Make AppRun aware of its parent directory
tresf Jul 9, 2017
0f99d4e
Typo
tresf Jul 9, 2017
040cc3c
Remove unecessary comment
tresf Jul 9, 2017
d4fe98a
Fix zyn linking attempt, misc cleanup
tresf Jul 9, 2017
8bea395
Add AppDir/usr/lib to LD_LIBRARY_PATH too
tresf Jul 9, 2017
62a57ff
Try to fix icon
tresf Jul 10, 2017
31fb899
Misc cleanup
tresf Jul 10, 2017
fc9f0f1
More cleanup
tresf Jul 10, 2017
6141c4d
Fix confusing ordering
tresf Jul 10, 2017
c33c4e5
Fix shellcheck, cosmetic stuff
tresf Jul 11, 2017
bf9d74b
Prevent making image from /usr/local
tresf Jul 11, 2017
eb5e819
Fix mimetype icon, Deb[ug] flags
tresf Jul 11, 2017
803b3a0
Fix menu icon
tresf Jul 11, 2017
8d89fd7
Attempt fix mimetypes
tresf Jul 13, 2017
0da1e6b
Mime work
tresf Jul 13, 2017
b301b04
Attempt to fix Qt58 compilation
tresf Jul 15, 2017
2982940
unset LD_LIBRARY_PATH
tresf Jul 15, 2017
7207675
More C++11/Qt5.8 fixes
tresf Jul 15, 2017
1cf825e
Fix LADSPA on Windows
tresf Jul 15, 2017
cea0622
Clean papu
tresf Jul 15, 2017
45a9c43
Fix VirtualBox crash
tresf Jul 15, 2017
ee7b988
Use weak loading of libjack
tresf Jul 17, 2017
064beb5
Fix weakjack
tresf Jul 17, 2017
dca8f97
Fix download cache logic
tresf Jul 18, 2017
22b88a0
Fix shellcheck warnings
tresf Jul 18, 2017
d923fe6
Merge branch 'stable-1.2' into appimage
tresf Jul 24, 2017
64e04d1
New approach at Zyn and VST linking
tresf Jul 24, 2017
04a42f7
Locate missing wine libs
tresf Jul 24, 2017
350c4d7
Remove wine workaround
tresf Jul 24, 2017
4271498
Handle alternative wine
tresf Jul 24, 2017
7f5a9f6
Clean up wine dependencies
tresf Jul 25, 2017
408a08e
Better wine workaround messaging
tresf Jul 25, 2017
4b4e754
Fix unecessary dirname logic
tresf Jul 21, 2017
752b7fa
Force linuxdeployqt to use cmake's qmake version
tresf Jul 25, 2017
43f9850
Add wine prefix to cmake
tresf Jul 25, 2017
2552305
More wine linking fixes
tresf Jul 25, 2017
208575b
Even more wine fixes
tresf Jul 25, 2017
33647a0
Yet another wine fix
tresf Jul 25, 2017
0fed492
Remove LD_LIBRARY_PATH from launcher
tresf Jul 26, 2017
8241043
Fix Zyn
tresf Jul 26, 2017
4a1ec1b
Prevent linking of .exe.so files
tresf Jul 26, 2017
c5cc585
Use forked linuxdeployqt to avoid libwine.so.1
tresf Jul 27, 2017
2c06b0d
Fix LADSPA linking
tresf Jul 27, 2017
a5ea5e9
Fix shellcheck warnings
tresf Jul 27, 2017
720a637
Consistent cmake message
tresf Jul 28, 2017
762e35b
Switch to appimagetool
tresf Aug 1, 2017
c727d9d
Fix shellcheck
tresf Aug 1, 2017
0382b69
Add stk rawwaves
tresf Aug 3, 2017
8bc9e9d
Add support for HiDPI screens
tresf Sep 20, 2017
a553aac
Revert "Fix weakjack"
tresf Oct 16, 2017
596c776
Revert "Use weak loading of libjack"
tresf Oct 16, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .travis/linux..install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ PACKAGES="$PACKAGES libjack0"

if [ $QT5 ]; then
PACKAGES="$PACKAGES qtbase5-dev qttools5-dev-tools qttools5-dev"
# For signing AppImage files (linuxdeployqt is Qt5 only)
PACKAGES="$PACKAGES gnupg2"
else
PACKAGES="$PACKAGES libqt4-dev"
fi
Expand Down
17 changes: 17 additions & 0 deletions cmake/linux/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,20 @@ INSTALL(FILES lmms.png DESTINATION "${DATA_DIR}/pixmaps")
INSTALL(FILES lmms DESTINATION "${DATA_DIR}/menu")
INSTALL(FILES lmms.desktop DESTINATION "${DATA_DIR}/applications")
INSTALL(FILES lmms.xml DESTINATION "${DATA_DIR}/mime/packages")

# AppImage creation target
SET(APPIMAGE_FILE "${CMAKE_BINARY_DIR}/${CMAKE_PROJECT_NAME}-${VERSION}-linux-${CMAKE_SYSTEM_PROCESSOR}.AppImage")

CONFIGURE_FILE("package_linux.sh.in" "${CMAKE_BINARY_DIR}/package_linux.sh" @ONLY)

FILE(REMOVE "${APPIMAGE_FILE}")
ADD_CUSTOM_TARGET(removeappimage
COMMAND touch "${APPIMAGE_FILE}" && rm "${APPIMAGE_FILE}"
COMMENT "Removing old AppImage")
ADD_CUSTOM_TARGET(appimage
COMMAND chmod +x "${CMAKE_BINARY_DIR}/package_linux.sh"
COMMAND "${CMAKE_BINARY_DIR}/package_linux.sh"
COMMENT "Generating AppImage")
ADD_DEPENDENCIES(appimage removeappimage)

# see also ../postinstall/CMakeLists.txt
95 changes: 95 additions & 0 deletions cmake/linux/package_linux.sh.in
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
#!/usr/bin/env bash
# Creates Linux ".AppImage" for @PROJECT_NAME_UCASE@
Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you going through all the hassle of making this part of CMake when a few lines of .travis.yml or shell script would do? Just curious. Do you think it would be possible/advantageous to have a generic CMake AppImage module/plugin? If so, would you know where to start?

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you going through all the hassle of making this part of CMake when a few lines of .travis.yml or shell script would do?

I hope I explained this (same?) question properly in #3688 (comment).

Do you think it would be possible/advantageous to have a generic CMake AppImage module/plugin? If so, would you know where to start?

Hesitant "yes" to "advantageous ... cmake module" question.

A "no" to knowing where to start. The first that comes to mind is CMake's NSIS integration, but it's really ugly to use.

CMake also offers CPackDMG for Mac, but it wasn't really useful for us. Our project requires macdeployqt combined with manual relinking. So from my perspective it's much easier to test, troubleshoot and patch in a common scripting language (which is bash for the time being)


set -e

# Console colors
RED="\\x1B[1;31m"
GREEN="\\x1B[1;32m"
YELLOW="\\x1B[1;33m"
PLAIN="\\x1B[0m"

USERBIN="$HOME/bin"
LINUXDEPLOYQT="$USERBIN/linuxdeployqt"
APPDIR="@CMAKE_BINARY_DIR@/@[email protected]/"

# TODO: Since DESTDIR is set at build time, the true install location is unknown.
INSTALL="@CMAKE_BINARY_DIR@/../target"

# linuxdeployqt assumes installations contain /usr/ prefixes.
# 1. Make a directory containing /usr
Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Step 1 should not be necessary.

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain.

Copy link

@probonopd probonopd Jul 8, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you install your build products with make INSTALL_ROOT="$APPDIR" install then they will end up there. The needed directories will be created for you.

Copy link
Member Author

@tresf tresf Jul 8, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First, you make mention of INSTALL_ROOT but your wiki references DESTDIR. Is there a difference?

# quoted from wiki
CMake wants DESTDIR instead:

  - cmake . -DCMAKE_INSTALL_PREFIX=/usr
  - make -j$(nproc)
  - make DESTDIR=appdir install ; find appdir/

Second, I did wonder if the parent directory would be created. We normally use cmake flags, as then we can use them in our scripts as well. If we can specify this through cmake instead, then we can configure it in and not have to guess. (Problem 2 above).

# > mkdir -p ../target/usr
# 2. Configure the install prefix for /usr only without "target"
# > cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DWANT_QT5=True
# 3. Use the DESTDIR variables to install to "target" instead
# > make DESTDIR=../target/ -j4 install
make DESTDIR=../target/ -j4 install
case "@CMAKE_INSTALL_PREFIX@" in
*/usr);;
*) echo -e "${RED}ERROR: linuxdeployqt requires install prefix of /usr:
Needs: mkdir ../target/usr
Needs: CMAKE_INSTALL_PREFIX=/usr
Needs: make DESTDIR=../target

Wrong: CMAKE_INSTALL_PREFIX=@CMAKE_INSTALL_PREFIX@
${PLAIN}"; exit 1;;
esac

mkdir -p "$USERBIN"

# Fetch portable linuxdeployqt
# TODO: Cache and only updated as needed
echo -e "${GREEN}Downloading linuxdeployqt to ${LINUXDEPLOYQT}${PLAIN}..."
wget https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage -O "$LINUXDEPLOYQT" -q
chmod +x "$LINUXDEPLOYQT"
echo " [success]"

# Make skeleton AppDir
Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think this is needed? When you install your build products with make INSTALL_ROOT="$APPDIR" install then they will end up there.

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is post-install step, so it's irrelevant.

Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this comment, why are you doing it when it is irrelevant?

Copy link
Member Author

@tresf tresf Jul 8, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your comment is irrelevant. You're telling a project how to specify the install target, which is too late when it's a post-install step. We can script the entire thing like we do for Windows, but then it takes away the granularity of developers specifying their own build environment.

We copy the install instead of messing it all up. Ideally, the /usr mandate would be dropped and then we can use whatever folder we want. That's how macdeployqt works for us.

It's really not your place to tell a project, or a user for that matter where to drop an install target. This may make sense for a small project. But (I feel) for large, complex projects, it's presumptuous.

Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So where would you ideally like to install? The reasonis that (at least for some software), the path specified with DCMAKE_INSTALL_PREFIX ends up in the compiled binaries, which is what I generally want to avoid, hence recommending /usr as the generic prefix. This is not a hard requirement though and you are free to use another prefix.

Copy link
Member Author

@tresf tresf Jul 8, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where would you ideally like to install

Wherever the user says to, would be the ideal location. For our tutorials, this is usually /home/user/lmms/target, but I often run a second copy from /home/user/Desktop/lmms/target and others may prefer their own place.

DCMAKE_INSTALL_PREFIX ends up in the compiled binaries, which is what I generally want to avoid

This is the part that confuses me... I would expect a relinking tool to be agnostic of the prefix, but to be fair, I've also never written one. :)

echo -e "${GREEN}Creating ${APPDIR}...${PLAIN}"
rm -rf "$APPDIR"
mkdir -p "$APPDIR"
echo " [success]"

# Clone install to AppDir
Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think this is needed? When you install your build products with make INSTALL_ROOT="$APPDIR" install then they will end up there.

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a post-install step, so it's irrelevant.

Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this comment, why are you doing it when it is irrelevant?

echo -e "${GREEN}Copying ${INSTALL} to ${APPDIR}...${PLAIN}"
cp -R "$INSTALL/usr" "$APPDIR"
echo " [success]"

# Prepare the launcher
Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think this is needed? linuxdeployqt will copy the desktop file and icon when you invoke it with linuxdeployqt $APPDIR/usr/share/applications/*.desktop

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per portablelinuxapps.com, I thought that was the way to do it.

Create an AppDir structure that looks (as a minimum) like this:

MyApp.AppDir/
MyApp.AppDir/AppRun
MyApp.AppDir/myapp.desktop
MyApp.AppDir/myapp.png
MyApp.AppDir/usr/bin/myapp
MyApp.AppDir/usr/lib/libfoo.so.0

Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That documentation describes how to do things by hand, it is from 2010 (7 years old). Nowadays we have tools like linuxdeployqt and appimagetool which can do the work.

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nowadays we have tools like linuxdeployqt and appimagetool which can do the work.

I started researching the layout when I had initial problems and there was no argument against it in #3558 so I thought I was on the right track. But we'll switch to using the recommended approach instead, thanks. 👍

echo -e "${GREEN}Setting up shortcuts...${PLAIN}"
mv "$APPDIR/usr/share/applications/@[email protected]" "$APPDIR"
mv "$APPDIR/usr/share/pixmaps/@[email protected]" "$APPDIR"
echo " [success]"

echo -e "${GREEN}Cleaning up...${PLAIN}"
Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't do this. $APPDIR/usr/share/applications/ and $APPDIR/usr/share/icons/ shall still be populated because that is what the optional appimaged daemon will use to integrate your application into the desktop (menus, icons, etc.)

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

# Cleanup orphaned directories
rmdir "$APPDIR/usr/share/applications/"
rmdir "$APPDIR/usr/share/pixmaps/"
echo " [success]"

# Workaround per https://github.com/probonopd/linuxdeployqt/issues/52
# export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/pulseaudio/:$LD_LIBRARY_PATH
# ^--- Shouldn't be needed anymore via https://github.com/probonopd/linuxdeployqt/commit/9a93e030cff2cb10e9e672ebb3360d30b3099c6d
export LD_LIBRARY_PATH="$APPDIR/usr/lib/lmms/":$LD_LIBRARY_PATH
Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed should not be needed anymore, did you try without this?

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet. It's on the todo list.

Copy link
Member Author

@tresf tresf Jul 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried it, still fails. Re-enabling.

ERROR: ldd outputLine: "libZynAddSubFxCore.so => not found" 
ERROR: Please ensure that all libraries can be found by ldd. Aborting. 
make[3]: *** [cmake/linux/CMakeFiles/appimage] Error 1
make[2]: *** [cmake/linux/CMakeFiles/appimage.dir/all] Error 2
make[1]: *** [cmake/linux/CMakeFiles/appimage.dir/rule] Error 2
make: *** [appimage] Error 2


echo -e "${GREEN}Bundling dependencies...${PLAIN}"
# Bundle both qt and non-qt dependencies into appimage format
"$LINUXDEPLOYQT" "$APPDIR/usr/bin/@CMAKE_PROJECT_NAME@" -executable="$APPDIR/usr/lib/@CMAKE_PROJECT_NAME@/RemoteZynAddSubFx" -bundle-non-qt-libs
Copy link

@probonopd probonopd Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move it to $APPDIR/usr/bin/RemoteZynAddSubFx and retry. I think linuxdeployqt fails because it expects executables to be in usr/bin and not in a subdirectory of usr/lib.

Copy link
Member Author

@tresf tresf Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not where it lives, so it will be more work. We can try it.

Copy link
Member Author

@tresf tresf Jul 8, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to clarify a bit on this... RemoteZynAddSubFx is not a binary usable by the system, so it's not something we'd ever install to $PATH. That's why it doesn't live there.

We have a similar problem with this executable on Mac and I haven't figured it out. Mac too makes mention that executables should be in a specific directory for macdeployqt to work properly, it's just not as straightforward as moving it. It may work for this PR, but it wouldn't make sense in general and when the remote process is launched from C++ it would need to know the new place to find it as well.

echo " [success]"

echo -e "${GREEN}Finishing the AppImage...${PLAIN}"
# Run a second time with appimage flag to fix liking
# TODO: This was reported as a linuxdeployqt "known issue" and can be removed once fixed
# https://github.com/probonopd/lmms/commit/dc9fd086e344e842613ab45623ed96246ab58e77#commitcomment-22187275
"$LINUXDEPLOYQT" "$APPDIR/usr/bin/@CMAKE_PROJECT_NAME@" -executable="$APPDIR/usr/lib/@CMAKE_PROJECT_NAME@/RemoteZynAddSubFx" -appimage
echo " [success]"


echo -e "${GREEN}Moving to @APPIMAGE_FILE@...${PLAIN}"
mv *.AppImage "@APPIMAGE_FILE@"

# Manually fix RemoteZynAddSubFx Qt5 linking
# FIXME: RemoteZynAddSubFx will still be linked to the system version e.g. /opt/qt5
# and will crash if launched on system without Qt5 natively installed.

echo -e "${GREEN}Finished${PLAIN}"