Skip to content

Develop#498

Merged
chrizandr merged 32 commits intomasterfrom
develop
Aug 31, 2020
Merged

Develop#498
chrizandr merged 32 commits intomasterfrom
develop

Conversation

@chrizandr
Copy link
Copy Markdown
Member

Fixes #

Checklist

  • My branch is up-to-date with upstream/develop branch.
  • Everything works and tested for Python 3.5.2 and above.

Description

Change logs

gustavodemorais and others added 30 commits October 12, 2019 21:54
* Version bump to 0.3.1

* Switching to manual deployment

* Update email

* Duplicated README (#425)

* Update chat room url to slack workspace.

Organization has moved from gitter to slack.
* Refactor checkClassOp() in resources.py

related to #424

* Fix failing imports

* Fix pep8 compliance

* Fix build errors

* Refactor post method in Item class

* Refactor put method in item class

* Refactor delete method in Item class

* Fix PEP8 issues
* docstring review changes

* docstring review changes

* docstring review changes FINAL

* test files

* test files added back

* itemcollection correction
* FIX: Incomplete contexts in hydrus demo

Add context to the following properties: `totalItems`,
`view`, `first`, `last`.

related to #429.

* Regenerate the API Documentation for the hydrus demo

Fixed incomplete contexts in API documentation of hydrus demo
and regenerated it.

* Fix tests with respect to context changes made

* Fix PEP8 issues

* Fix PEP8
add support for .py, .yaml, .jsonld in environment variable

reformat code

move logger info below else statement

add base exception in conf.py

refactor code using functions

improve comments

reduce line length-PEP8

add Raise keyword in docstring
* fixed bug in cli.py

* update cli.py

* pep8 compliant
* Replace authetication of request with decorator

* refactor

* storing the item object in item_class thereby reducing the rather
expensive call of get_doc() to get title and path separately
* remove unused variable in function
* remove unnecessary else statement

* Refactor resources.py

* move validation logic for an object before it is inserted/updated
into separate function called `validate_object` in helpers.py
* reduce the function call of `get_doc` while generating the collections
and parsed classes of a api_doc and moved it to a separate function

* Refactor resources.py

* Abstract the logic of response creation to more lower level functions
instead of keeping it in resources.py increasing readability. Now resources.py
only imports required helper functions instead of importing a lot of utility
functions and exception classes.
* get_context function abstacts away logic of dynamically generating response.
* error_response function generates responses for errors during operations
* send_update function is handler for sending synchronization update to all
connected clients
* create item_collection_helpers.py to handle the response generation
of GET, PUT, POST and DELETE operations on Item **Collection** classes.
* create items_helpers.py to handle the response generation
of PUT and DELETE operation on **multiple** items.
Fix headers_ variable under POST method of Item Collection Resource.
fixes #462
* Use error_response and validate_object functions

* Convert string formats to use f strings in the whole codebase

* Update travis config to drop support for python < 3.6

* Update README

* hydrus drops support for python < 3.6

* Removed build for python 3.8 due to issues with psycopg2

Refer: psycopg/psycopg2#858

* Bring back build for python 3.6-dev and 3.7-dev
Removed 'flask-python3.5-index-upload'
* Update them with the info that hydrus only supports python version >= 3.6.0
* Update Dockerfile

Removed 'flask-python3.5-index-upload'

* Added cli option to use existing database

* Fixed pep8
* Migrate test suite from unittest to pytest - 1

* Migrate authentication tests from unittest to pytest
  in tests/functional/test_auth.py
* Delete tests/test_auth.py

* Migrate test suite from unittest to pytest - 2

* Migrate CRUD operation tests from unittest to pytest
  in tests/unit/test_crud.py
* Delete tests/test_crud.py

* Migrate test suite from unittest to pytest - 3

* Migrate app tests and socket tests from unittest to pytest
  in tests/functional/test_app.py and
  tests/functional/test_socket.py respectively
* Delete tests/test_app.py

* Migrate test suite from unittest to pytest - 4

* Migrate the test_cli.py and test_pep8.py in /hydrus/tests
to pytest

* Update travis config to run pytest tests instead of unittest

* Add pytest to requirements.txt

* Fix PEP8 compliance

* Fix PEP8

* Deleted old test files
The code was using the old hydrus api. Update it.
Using the previous base image, tiangolo/uwsgi-nginx-flask, gave
error with server certificate verification.
Refer tiangolo/uwsgi-nginx-flask-docker#181 for more details.

Also, switched base image from python3.8 to python3.6 as psycopg2 as error on building
on python3.8.
Refer psycopg/psycopg2#858 for more details.

Closes #473
Made the default PORT and API_NAME same for both the cases,
when hydrus is started from `hydrus serve` as well
`docker-compose up --build`.

Now default PORT is 8080 and API_NAME is "serverapi"
* Add get_rdf_class() function

Replace repeating code for finding RDF class from type_ with get_rdf_class()
function.

* Add get_single_instance and get_data_iac_iii_iit functions

* Add add_prop_name_to_object function

* Add get_instance_before_delete function

* Refactor get_collection function

* Add docstrings of all new / refactored functions

* Refactored insert_multiple function to use insert function

* Refactored delete_multiple function to use delete function

* Fix insert_multiple function
* Make db tables for each resource in apidoc

* Add functionality to insert to db

* Add CRUD interface

* Add Foreign key links

* Add Exception Handling

* Add comments and docstrings

* cleaned code

* Integrate in existing hydrus framework

* Add support for insert on nested object

* Make copy for object before insert

* Modify conftest to initialise db correctly for new schema

* Include hydra:Link property while making dummy objects

* Fix typo

* Update some tests so they match newer insert functions api

* Fix get_single, update_single, delete_single and insert_single to use new database schema

* Initialise db correctly for running socket unit tests

* Initialise db correctly for running auth unit tests

* Remove unnecessary pdb import

* Fix pagination bug in new database schema

* Fix database init for app tests

* Fix PEP8 warnings

* Remove unnecessary code from previous database schema

* Fix tests failure

The same pytest setup fixture being run multiple times for setup for
different modules resulted in same tables being
added to the same 'sqlalchemy.ext.declarative.declarative_base'
instance. This resulted error from sqlalchemy as the table was
already defined in that MetaData instance.

* Add doc strings in resource_based_classes.py

* Add doc strings in db_models.py

* Remove unused function

* Remove unnecessary old code

* Add type hints

* Fix PEP8 warnings

* Remove unused commented code

* Remove unnecessary code

* Move pagination to another function

* Add support for nested search params

* Catch specific exceptions while filtering

* Catch specific exception while deleting

* Add exception handling while getting @type of object

* Remove unnecessary DatabaseConstraintError Exception

We are already catching all errors related to foreign key
before session.commit(). Also we are adding the foreign keys
before insert. So no exception related to database constraint
should be raised by sql alchemy on session.commit().

* Add tests for the added new exception(PropertyNotGiven)

* Fix pep8 errors

* Treat HydraLink as just links, not as resources

Removed the dependency of treating HydraLink as foreign key
columns. Now treating them like normal columns

* Fix pep8

* Use PropertyNotGiven exception instead of generic exception
* Remove hardcoding of "Collection" keyword

* Remove dependency of collection word in crud.py

All the resources uris for collections will now be made just from
path variable information. Their will not be dependency
of 'type_' variable on resource uri. As the 'type_' value
might be something and the actual 'path' for that collection
can be different. It need not be always 'type_Collection'

* Fix @type in pagination

* Fix the condition for non-collection classes
* Add collection_id column for collection classes

* Shift GET and PUT operations for non collection classes

* Support /classname/:id endpoint on non collection classes

* Support GET on /collection endpoint

* Support PUT operation on /collection endpoint

* Remove old DELETE endpoint for /collection

No need of old DELETE endpoint for /collection as that actually
was used for non-collection classes.

* Remove old POST endpoint for /collection

That was actually used for non-collection classes but we don't need
that now as now(after implementing collection as a resource),
non-collection classes have same behaviour as normal classes which
become part of collection

* Fix wrong collection name and type_

* Support GET on /collection/:uuid endpoint

Before /collection/:uuid would get all the details of the object
instance with id as uuid and type_ of the class of the collection.

Now, that endpoint will return all the members of the collection
with collection_id as that uuid.

* Support PUT on /collection/:uuid endpoint

* Support DELETE on /collection/:uuid endpoint

* Correct condition for NoResultFound Exception

* Fix wrong collection condition

* Support PUT operation on /collection/:uuid endpoint

* Default collection argument to False

* Fix auth tests for new endpoints

POST is not allowed on collection endpoints.
It was allowed before for the case when the class endpoint is
of a class which does not form a collection. But in the new
endpoint system, there is no need of this differentiation as
a class which does not form a collection and which does form a collection,
both have same behavior with respect to endpoints.

So changed POST to PUT.

* Fix socket tests

Now POST and PUT operation for any class happens at
/classname/:uuid endpoint

* Support added for collections

gen_dummy_object generates correct object for collections types now

also, temporarily add manages block explicitly to collection-classes
until appropriate changes take place in hydra-python-core library.

* Fix app tests for new collection endpoints

* Fix PEP8 issues

* Add tests for new collection endpoints

* Pass collection argument to get_object function
* Use expanded @id for EntryPoint in doc_parse function

* Get resource name from expanded @id

* Get foreign key relationships through expanded ids

* get vocab route from apidoc to build vocab endpoint

* remove hardcoding of vocab keyword on link header

* remove hardcoding of vocab keyword in checkEndpoint function

* Fix hardcoding of vocab in getType function

* Fix hardcoding of vocab keyword in finalize_response function

* Fix hardcoding of vocab keyword in generate_iri_mappings

* Fix hardcoding of vocab keyword in get_link_props

* Remove hardcoding of vocab keyword in conftest.py

* Fix hardcoded vocab - test_app.py

* Remove hardcoded vocab - test_curd.py

* Get the parsed_class name directly

* Remove accidental import of pdb

* Use correct conditions for propertyNotGiven test

* Fix tests as now collections array in entrypoint response

* Correctly get mananged class type and title for collections

* Fix multiple add and delete endpoints for parsed classes

* Fix tests for updates in the hydra-python-core repo

Fix tests for collections array in EntryPoint endpoint response
(HTTP-APIs/hydra-python-core#40).
Fix tests for expanded ids.

* Add check for db setup

* Add check for PUT operation before inserting objects

* Fix IRI template tests

* Revert "Add check for db setup"

This reverts commit e152fa2.

* Update doc_writer_sample.py and its output with latest changes at core library

* Update Drone doc with latest changes at core

* Fix PEP8 errors
* update docs to the latest core library

* update doc_gen
* Add member_type column in for collection tables

* Add helper function for getting type from path

* Support PUT for multiple class collection

* Support GET for multiple class collection

* Add helper function to check if instance exists

* Check member instance in PUT for collection

* Add PUT support for multiple class collection at /collection/:uuid endpoint

* Refactor function for parsing collection members

* Add POST support for multiple class collection at /collection/:uuid endpoint

* Correct variable name usage

* Generate dummy object in correct format to support multiple class collections

* Changed format of request body for PUT at /collection endpoint

* Check manages block of collection while generating dummy object
* Add docs on database of hydrus

* Minor typos

* Add sample ER diagram
* previously /class endpoint served all the members of class,
now made /collection endpoint to serve all members of class.
* /class endpoint still used for adding class members.(PUT)
* GET on /class endpoint gives 405
@sameshl
Copy link
Copy Markdown
Member

sameshl commented Aug 31, 2020

@chrizandr Fixed the merge conflict. PR can be merged

@chrizandr chrizandr merged commit 08b2a57 into master Aug 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants