Skip to content

Tags: TheJokersThief/daft-scraper

Tags

1.4.0

Toggle 1.4.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fix three issues I faced when using the scraper (#35)

* Update USER_AGENT for scraper

To mitigate denied error.

* Update images in ListingMedia(Schema)

This change mitigate the following issue:

```python
Traceback (most recent call last):
File "/Users/myuser/Documents/daft-main/main.py", line 49, in <module>
for listing in listings:
^^^^^^^^
File "/Users/myuser/Documents/daft-main/.venv/lib/python3.12/site-packages/daft_scraper/search/init.py", line 57, in search
yield from self._get_listings(listing_data)
File "/Users/myuser/Documents/daft-main/.venv/lib/python3.12/site-packages/daft_scraper/search/init.py", line 92, in _get_listings
Listing(ListingSchema().load(listing['listing'])),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/myuser/Documents/daft-main/.venv/lib/python3.12/site-packages/marshmallow/schema.py", line 792, in load
return self._do_load(
^^^^^^^^^^^^^^
File "/Users/myuser/Documents/daft-main/.venv/lib/python3.12/site-packages/marshmallow/schema.py", line 999, in _do_load
raise exc
marshmallow.exceptions.ValidationError: {'media': {'images': {0: defaultdict(<class 'dict'>, {'imageLabels': {'value': ['Not a valid string.']}}), 2: defaultdict(<class 'dict'>, {'imageLabels': {'value': ['Not a valid string.']}}), 4: defaultdict(<class 'dict'>, {'imageLabels': {'value': ['Not a valid string.']}})}}}
```

* Update Page Logic

`from = <value>` no longer works,
Update the search to use `page = <value>`

Have tested in my local env.
Before updates the scrapper would keep load same page for 50 times..

1.3.1

Toggle 1.3.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Bump urllib3 from 1.26.18 to 1.26.19 (#29)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.18 to 1.26.19.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/1.26.19/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.18...1.26.19)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

1.3.0

Toggle 1.3.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
DAFT-15: Allow an initial offset (#17)

1.2.7

Toggle 1.2.7's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
DAFT-14: Add new property types for sales (#16)

1.2.6

Toggle 1.2.6's commit message
Bugfix: Prevent listing fetching for every ID by specifying data_key …

…in schema

1.2.5

Toggle 1.2.5's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
DAFT-13: Add a fallback for fetching the ID (#14)

1.2.4

Toggle 1.2.4's commit message
Bump version after DAFT-12

1.2.3

Toggle 1.2.3's commit message
Bump version for DAFT-11

1.2.2

Toggle 1.2.2's commit message
Bump version to fix coords float bug

1.2.1

Toggle 1.2.1's commit message
Add missing point data