Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
bcfb173
attachments are saved as intact files
dimitri-yatsenko Sep 13, 2019
18366e6
reform attachments and filepath
dimitri-yatsenko Sep 13, 2019
11d7d9b
complete implementation of external storage except for `clean`
dimitri-yatsenko Sep 16, 2019
22347ae
Merge branch 'dev' of https://github.com/datajoint/datajoint-python i…
dimitri-yatsenko Sep 16, 2019
7e7c183
refactor external storage
dimitri-yatsenko Sep 18, 2019
8aa1543
complete refactoring of external storage for version 0.12
dimitri-yatsenko Sep 20, 2019
a2d6f9a
rename attribute `basename` to `attachment_name` in external table
dimitri-yatsenko Sep 20, 2019
2c11a65
add __repr__ to ExternalMapping
dimitri-yatsenko Sep 20, 2019
3a1c6ab
external files are not copied if stage and store are the same
dimitri-yatsenko Sep 20, 2019
478b36a
make external tables require setting the `delete_external_files` argu…
dimitri-yatsenko Sep 20, 2019
bb1deab
update CHANGELOG
dimitri-yatsenko Sep 23, 2019
e2636ed
fix Python 3.4 compatibility
dimitri-yatsenko Sep 23, 2019
11a1f93
fix Python 3.4 and 3.5 compatibility
dimitri-yatsenko Sep 23, 2019
165f795
dropped support for Python 3.4
dimitri-yatsenko Sep 23, 2019
c18af74
Merge branch 'dev' of https://github.com/datajoint/datajoint-python i…
dimitri-yatsenko Sep 26, 2019
48147f1
minor changes in error messages
dimitri-yatsenko Oct 1, 2019
2e26c4a
Merge branch 'dev' of https://github.com/datajoint/datajoint-python i…
dimitri-yatsenko Oct 3, 2019
0c4fd1c
Update to pathlib in test init.
guzman-raphael Oct 3, 2019
6ba86e0
Update test_blob_migrate to be compatible for WIN10.
guzman-raphael Oct 3, 2019
43a5126
Fix WIN10 compatibility with KeyboardInterrupt and SystemExit excepti…
guzman-raphael Oct 3, 2019
c7ca34c
Merge pull request #4 from guzman-raphael/dimitri-attach
dimitri-yatsenko Oct 3, 2019
5c99e37
Fix WIN10 filepath to store as posix and fetch as user's platform.
guzman-raphael Oct 4, 2019
e2c3f23
Fix relpath for Python3.5.
guzman-raphael Oct 4, 2019
93aeefc
Fix copytree for Python3.5.
guzman-raphael Oct 4, 2019
20719ae
Fix typo.
guzman-raphael Oct 4, 2019
7ca0099
Fix for Python3.5.
guzman-raphael Oct 4, 2019
f3ffd63
Update coveralls.
guzman-raphael Oct 4, 2019
bb1b40f
Update coverall env vars.
guzman-raphael Oct 4, 2019
f05c50d
Merge pull request #5 from guzman-raphael/win-filepath
dimitri-yatsenko Oct 5, 2019
298efed
add environment variable DJ_SUPPORT_FILEPATH_MANAGEMENT to enable/dis…
dimitri-yatsenko Oct 7, 2019
e609fbe
Merge branch 'attach' of https://github.com/dimitri-yatsenko/datajoin…
dimitri-yatsenko Oct 7, 2019
6dda528
Update CHANGELOG.md
dimitri-yatsenko Oct 8, 2019
796dae6
Update CHANGELOG.md
dimitri-yatsenko Oct 8, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
dropped support for Python 3.4
  • Loading branch information
dimitri-yatsenko committed Sep 23, 2019
commit 165f795726eb282b58bbaee55b006d8e52506fe0
1 change: 0 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ language: python
env:
- DJ_TEST_HOST="127.0.0.1" DJ_TEST_USER="datajoint" DJ_TEST_PASSWORD="datajoint" DJ_HOST="127.0.0.1" DJ_USER="root" DJ_PASS="" S3_ACCESS_KEY="datajoint" S3_SECRET_KEY="datajoint"
python:
- "3.4"
- "3.5"
- "3.6"
- "3.7"
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
## Release notes

### 0.12.0 -- Aug 23, 2019
* Dropped support for Python 3.4
* Support TLS/SSL connections PR #620
* Convert numpy array from python object to appropriate data type if all elements are of the same type (#587) PR #608
* Remove expression requirement to have additional attributes (#604) PR #604
Expand Down
6 changes: 2 additions & 4 deletions datajoint/external.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,7 @@ def _download_buffer(self, external_path):
if self.spec['protocol'] == 's3':
return self.s3.get(external_path)
if self.spec['protocol'] == 'file':
with Path(external_path).open('rb') as f:
return f.read()
return Path(external_path).read_bytes()
assert False

def _remove_external_file(self, external_path):
Expand Down Expand Up @@ -157,8 +156,7 @@ def get(self, uuid):
try:
cache_path = Path(cache_folder, *subfold(uuid.hex, CACHE_SUBFOLDING))
cache_file = Path(cache_path, uuid.hex)
with cache_file.open('rb') as f:
blob = f.read()
blob = cache_file.read_bytes()
except FileNotFoundError:
pass # not cached
# download blob from external store
Expand Down
3 changes: 1 addition & 2 deletions datajoint/table.py
Original file line number Diff line number Diff line change
Expand Up @@ -277,8 +277,7 @@ def make_placeholder(name, value):
value = self.external[attr.store].upload_attachment(attachment_path).bytes
else:
# value is filename + contents
with attachment_path.open('rb') as f:
value = str.encode(attachment_path.name) + b'\0' + f.read()
value = str.encode(attachment_path.name) + b'\0' + attachment_path.read_bytes()
elif attr.is_filepath:
value = self.external[attr.store].upload_filepath(value).bytes
elif attr.numeric:
Expand Down
3 changes: 1 addition & 2 deletions datajoint/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,7 @@ def safe_write(filepath, blob):
if not filepath.is_file():
filepath.parent.mkdir(parents=True, exist_ok=True)
temp_file = filepath.with_suffix(filepath.suffix + '.saving')
with temp_file.open('wb') as f:
f.write(blob)
temp_file.write_bytes(blob)
temp_file.rename(filepath)


Expand Down