Skip to content

[SPARK-13474][PROJECT INFRA] Update packaging scripts to push artifacts to home.apache.org #11350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JoshRosen wants to merge 15 commits into from
Closed

[SPARK-13474][PROJECT INFRA] Update packaging scripts to push artifacts to home.apache.org #11350

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
4d07db4
Chages for people.apache.org -> home.apache.org migration.
JoshRosen Feb 24, 2016
a458221
Remove auto-confirm.
JoshRosen Feb 24, 2016
3a7b283
Configure SSH options for lftp.
JoshRosen Feb 24, 2016
19d6e22
Allow parameter substitution in COMMANDS heredoc.
JoshRosen Feb 24, 2016
e43f7ac
mput syntax fix.
JoshRosen Feb 24, 2016
1bb9e02
use rm -f
JoshRosen Feb 25, 2016
52fedd0
Fixes to removal of old directories.
JoshRosen Feb 25, 2016
fc80ac7
Reverse sort order.
JoshRosen Feb 25, 2016
1aed97e
Try to work around lack of symlink support in FTP.
JoshRosen Feb 25, 2016
fde8346
Debug flags.
JoshRosen Feb 25, 2016
b73a471
Exit 0 in rm.
JoshRosen Feb 25, 2016
9ef4561
Let lftp do glob expansion.
JoshRosen Feb 25, 2016
36c3466
&& -> ||
JoshRosen Feb 25, 2016
e448ae6
Try using put instead of mput for docs.
JoshRosen Feb 26, 2016
dd2766a
Try using lftp mirror instead
JoshRosen Feb 26, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Exit 0 in rm.
  • Loading branch information
@JoshRosen
JoshRosen committed Feb 25, 2016
commit b73a47117e066ecf0569c90b3882a82210f76824
8 changes: 3 additions & 5 deletions dev/create-release/release-build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ if [ -n "$REMOTE_PARENT_MAX_LENGTH" ]; then
| tail -n +$REMOTE_PARENT_MAX_LENGTH)
for old_dir in $old_dirs; do
echo "Removing directory: $old_dir"
LFTP rm -rf $REMOTE_PARENT_DIR/$old_dir
LFTP "rm -rf $REMOTE_PARENT_DIR/$old_dir && exit 0"
done
fi

Expand Down Expand Up @@ -198,17 +198,15 @@ if [[ "$1" == "package" ]]; then
# Copy data
dest_dir="$REMOTE_PARENT_DIR/${DEST_DIR_NAME}-bin"
echo "Copying release tarballs to $dest_dir"
set -x
# Put to new directory:
LFTP mkdir -p $dest_dir
LFTP mput -O $dest_dir spark-*
# Delete /latest directory and rename new upload to /latest
LFTP rm -rf "$REMOTE_PARENT_DIR/latest"
LFTP "rm -r -f $REMOTE_PARENT_DIR/latest && exit 0"
LFTP mv $dest_dir "$REMOTE_PARENT_DIR/latest"
# Re-upload a second time and leave the files in the timestamped upload directory:
LFTP mkdir -p $dest_dir
LFTP mput -O $dest_dir spark-*
Copy link
Contributor Author

@JoshRosen JoshRosen Feb 25, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might need to quote the spark-* so that the parameter expansion is done by lftp rather than the shell.

set +x
exit 0
fi

Expand All @@ -226,7 +224,7 @@ if [[ "$1" == "docs" ]]; then
LFTP mkdir -p $dest_dir
LFTP mput -O $dest_dir _site/*
# Delete /latest directory and rename new upload to /latest
LFTP rm -rf "$REMOTE_PARENT_DIR/latest"
LFTP "rm -r -f $REMOTE_PARENT_DIR/latest && exit 0"
LFTP mv $dest_dir "$REMOTE_PARENT_DIR/latest"
# Re-upload a second time and leave the files in the timestamped upload directory:
LFTP mkdir -p $dest_dir
Expand Down