Skip to content

Conversation

@dhoomakethu
Copy link
Contributor

@dhoomakethu dhoomakethu commented Oct 7, 2019

  • Asyncio Server implementation (Python 3.7 and above only)
  • Bug fix for DiagnosticStatusResponse when odd sized response is received
  • Remove Pycrypto from dependencies and include cryptodome instead
  • Remove SIX requirement pinned to exact version.
  • Minor bug-fixes in documentations.

Pre-Release available @ https://pypi.org/project/pymodbus/2.3.0rc1/

dices and others added 30 commits September 28, 2018 02:30
…names

Updated requirements-docs.txt to include missing modules
Commenting out python3.7 from Travis while waiting for support. You can run teh 3.7 tests with tox without issues
Fixing endianness in Payload example.
…ev3.7

# Conflicts:
#	CHANGELOG.rst
#	doc/conf.py
#	pymodbus/framer/ascii_framer.py
#	pymodbus/framer/binary_framer.py
#	pymodbus/framer/rtu_framer.py
#	pymodbus/framer/socket_framer.py
#	pymodbus/repl/client.py
#	pymodbus/repl/completer.py
#	pymodbus/server/asynchronous.py
Dev3.7 async keyword changes
* check self.socket

self.socket might be None at this point

* Update pymodbus/client/sync.py

Co-Authored-By: mpf82 <[email protected]>
* #357 Support registration of custom requests

* #368 Fixes write to broadcast address

When writing to broadcast address (unit_id=0) there should be no response according to the Modbus spec. This fix changes expected_response_length to 0 when writing to unit_id=0. This will break any existing code that is improperly using unit_id 0 for a slave address.

* Bump version to 2.2.0

Fix #366 Update failures in sql context

Update Changelog

Fix major minor version in example codes

* Fix #371 pymodbus repl on python3

* 1. Fix tornado async serial client `TypeError` while processing incoming packet.
2. Fix asyncio examples.
3. Minor update in factory.py, now server logs prints received request instead of only function cod

* [fix v3] poprawa sprawdzania timeout

* Release candidate for pymodbus 2.2.0

*  Fix #377 when invalid port is supplied and minor updates in logging

* #368 adds broadcast support for sync client and server

Adds broadcast_enable parameter to client and server, default value is False. When true it will treat unit_id 0 as broadcast and execute requests on all server slave contexts and not send a response and on the client side will send the request and not try to receive a response.

* #368 Fixes minor bug in broadcast support code

* Fixed erronous CRC handling

If the CRC recieved is not correct in my case my slave got caught in a deadlock, not taking any new requests. This addition fixed that.

* Update Changelog

* Fix test coverage

* Fix #387 Transactions failing on 2.2.0rc2.

* Task Cancellation and CRC Errors

Alternate solution for #356 and #360.

Changes the RTU to make the transaction ID as the unit ID instead of an ever incrementing number.

Previously this transaction ID was always 0 on the receiving end but was the unique transaction ID on sending.

As such the FIFO buffer made the most sense. By tying it to the unit ID, we can recover from failure modes such as: -
- Asyncio task cancellations (eg. timeouts) #360
- Skipped responses from slaves. (hangs on master #360)
- CRC Errors #356
- Busy response

* Cherry pick commit from PR #367 , Update changelog , bump version to 2.2.0rc4

* #389 Support passing all serial port parameters to asynchronous server

* Fix BinaryPayloadDecoder and Builder wrt to coils

* Misc updates, bump version to 2.2.0

* ReportSlaveIdResponse now tries to get slave id based on server identity for pymodbus servers

* Update missing bcrypt requirement for testing
* Fix document generation

* Formatting updates in Changelog
It has not been needed by Twisted for a long time, and has been unmaintained
for a long time.
* Fix document generation

* Formatting updates in Changelog

* Fix --upgrade option in install dependencies
If the response is odd size the buffer needs to be padded with an additional byte.
Python 3.4 is EoL and has an easy upgrade path to 3.5+. Support was
dropped in Twisted 19.7.0, which is causing Travis to fail pymodbus
tests for 3.4.
tracernz and others added 7 commits September 9, 2019 16:52
* #357 Support registration of custom requests

* #368 Fixes write to broadcast address

When writing to broadcast address (unit_id=0) there should be no response according to the Modbus spec. This fix changes expected_response_length to 0 when writing to unit_id=0. This will break any existing code that is improperly using unit_id 0 for a slave address.

* Bump version to 2.2.0

Fix #366 Update failures in sql context

Update Changelog

Fix major minor version in example codes

* Fix #371 pymodbus repl on python3

* 1. Fix tornado async serial client `TypeError` while processing incoming packet.
2. Fix asyncio examples.
3. Minor update in factory.py, now server logs prints received request instead of only function cod

* [fix v3] poprawa sprawdzania timeout

* Release candidate for pymodbus 2.2.0

*  Fix #377 when invalid port is supplied and minor updates in logging

* #368 adds broadcast support for sync client and server

Adds broadcast_enable parameter to client and server, default value is False. When true it will treat unit_id 0 as broadcast and execute requests on all server slave contexts and not send a response and on the client side will send the request and not try to receive a response.

* #368 Fixes minor bug in broadcast support code

* Fixed erronous CRC handling

If the CRC recieved is not correct in my case my slave got caught in a deadlock, not taking any new requests. This addition fixed that.

* Update Changelog

* Fix test coverage

* Fix #387 Transactions failing on 2.2.0rc2.

* Task Cancellation and CRC Errors

Alternate solution for #356 and #360.

Changes the RTU to make the transaction ID as the unit ID instead of an ever incrementing number.

Previously this transaction ID was always 0 on the receiving end but was the unique transaction ID on sending.

As such the FIFO buffer made the most sense. By tying it to the unit ID, we can recover from failure modes such as: -
- Asyncio task cancellations (eg. timeouts) #360
- Skipped responses from slaves. (hangs on master #360)
- CRC Errors #356
- Busy response

* Cherry pick commit from PR #367 , Update changelog , bump version to 2.2.0rc4

* native asyncio implementation of ModbusTcpServer and ModbusUdpServer

* preliminary asyncio server examples

* move serial module dependency into class instantiation

* unittests for asyncio based server implementation

* induce exception in execute method by mock patching the request object's execute method

* move serial module dependency into class instantiation

* added asynctest depency to requirements-tests.txt

* add unittest skip condition for unsupported targets, remove failing assertion from unsupported targets, use lower asynctest version

* remove logger setLevel call since doing so may override library consumers' already set log level

* remove async def/await keywords from unittest so that the ast can be loaded in py2 even if the test is to be skipped
* repl: Allow Modbus RTU framing on a TCP socket

* repl: Update README for framing option
starnight and others added 5 commits October 17, 2019 22:14
* client/sync.py: Fix missing serial module dependency

The serial.connect failed in PR riptideio#400 with "NameError: name
'serial' is not defined" [1]:

self = <ModbusSerialClient at 0x7fcda4009b00 socket=None, method=ascii, timeout=3>

    def connect(self):
        """ Connect to the modbus serial server

        :returns: True if connection succeeded, False otherwise
        """
        if self.socket:
            return True
        try:
>           self.socket = serial.Serial(port=self.port,
                                        timeout=self.timeout,
                                        bytesize=self.bytesize,
                                        stopbits=self.stopbits,
                                        baudrate=self.baudrate,
                                        parity=self.parity)
E                                       NameError: name 'serial' is not defined

pymodbus/client/sync.py:476: NameError

This patch moves the serial import back to the head.

[1] https://travis-ci.org/riptideio/pymodbus/jobs/566009109

Fixes: commit e6da559 asyncio server implementation (#400)

* server/asyncio.py: Create server with appropriate args and environment

If Python is older than 3.7, the create_server will fail like PR
riptideio#400 with "unexpected keyword argument 'start_serving'" [1]
which is new in Python version 3.7:

self.server_factory = self.loop.create_server(lambda :self.handler(self),
                                              *self.address,
                                              reuse_address=allow_reuse_address,
                                              reuse_port=allow_reuse_port,
                                              backlog=backlog,
>                                             start_serving=not defer_start)

E       TypeError: create_server() got an unexpected keyword argument 'start_serving'

pymodbus/server/asyncio.py:400: TypeError

This patch creates server according to Python environment.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584178484

Fixes: commit e6da559 asyncio server implementation (#400)

* Create asyncio task with appropriate method and environment

If Python is older than 3.7, the asyncio.create_task will fail like PR
riptideio#400 with "AttributeError: module 'asyncio' has no attribute
'create_task'" [1] which is new in Python version 3.7 [2]:

@asyncio.coroutine

def testTcpServerCloseActiveConnection(self):
    ''' Test server_close() while there are active TCP connections'''
    data = b"\x01\x00\x00\x00\x00\x06\x01\x01\x00\x00\x00\x01"
    server = yield from StartTcpServer(context=self.context,address=("127.0.0.1", 0),loop=self.loop)
>   server_task = asyncio.create_task(server.serve_forever())
E   AttributeError: module 'asyncio' has no attribute 'create_task'

test/test_server_asyncio.py:205: AttributeError

This patch creates task according to Python environment.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584193587
[2] https://docs.python.org/3/library/asyncio-task.html#creating-tasks

Fixes: commit e6da559 asyncio server implementation (#400)

* server/asyncio.py: Fix format string for older Python

If Python is older than 3.6, f-Strings will fail like PR riptideio#400
with "SyntaxError: invalid syntax" [1] which is new in Python version
3.6 with PEP 498 -- Literal String Interpolation [2]:

test/test_server_asyncio.py:14: in <module>
    from pymodbus.server.asyncio import StartTcpServer, StartUdpServer, StartSerialServer, StopServer, ModbusServerFactory
E     File "/home/travis/build/starnight/pymodbus/pymodbus/server/asyncio.py", line 424
E       _logger.warning(f"aborting active session {k}")
E                                                    ^
E   SyntaxError: invalid syntax

This patch fixes the format string with traditional format string
syntax.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584427976
[2] https://www.python.org/dev/peps/pep-0498/

Fixes: commit e6da559 asyncio server implementation (#400)

* test: Make assert_called_once() test only with Python 3.6+

If Python is older than 3.6, unittest.mock.assert_called_once() will
fail like PR riptideio#400 with "AttributeError: assert_called_once" [1]
which is new in Python version 3.6 [2]:

>       self.loop.create_server.assert_called_once()

test/test_server_asyncio.py:76:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <CoroutineMock name='mock.create_server' id='139638313234784'>
name = 'assert_called_once'

    def __getattr__(self, name):
        if name in {'_mock_methods', '_mock_unsafe'}:
            raise AttributeError(name)
        elif self._mock_methods is not None:
            if name not in self._mock_methods or name in _all_magics:
                raise AttributeError("Mock object has no attribute %r" % name)
        elif _is_magic(name):
            raise AttributeError(name)
        if not self._mock_unsafe:
            if name.startswith(('assert', 'assret')):
>               raise AttributeError(name)
E               AttributeError: assert_called_once

/opt/python/3.5.6/lib/python3.5/unittest/mock.py:585: AttributeError

This patch skips the tests if they are not in Python 3.6+.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584431003
[2] https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called_once

Fixes: commit e6da559 asyncio server implementation (#400)

* test: Make serve_forever() test only with Python 3.7+

If Python is older than 3.7, asyncio.base_events.Server.serve_forever
will fail like PR riptideio#400 with "AttributeError: <class
'asyncio.base_events.Server'> does not have the attribute
'serve_forever'" [1] which is new in Python version 3.7 [2]:

@asyncio.coroutine
def testTcpServerServeNoDefer(self):
    ''' Test StartTcpServer without deferred start (immediate execution of server) '''
>   with patch('asyncio.base_events.Server.serve_forever', new_callable=asynctest.CoroutineMock) as serve:

test/test_server_asyncio.py:81:

...

if not self.create and original is DEFAULT:
    raise AttributeError(
>       "%s does not have the attribute %r" % (target, name)
    )
E   AttributeError: <class 'asyncio.base_events.Server'> does not have the attribute 'serve_forever'

This patch skips the tests if they are not in Python 3.7+.

[1] https://travis-ci.org/starnight/pymodbus/jobs/584212511
[2] https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.Server.serve_forever

Fixes: commit e6da559 asyncio server implementation (#400)
* Add TLS feature for Modbus synchronous

Modbus.org released MODBUS/TCP Security Protocol Specification [1],
which focuses variant of the Mobdbus/TCP protocol utilizing Transport
Layer Security (TLS). This patch enables the Modbus over TLS feature as
ModbusTlsClient with the Python builtin module ssl - TLS/SSL wrapper for
socket objects.

[1]: http://modbus.org/docs/MB-TCP-Security-v21_2018-07-24.pdf

* Implement MODBUS TLS synchronous server

Since we have the MODBUS TLS synchronous client, we can also have the
MODBUS TLS synchronous server.
@dhoomakethu dhoomakethu merged commit fbdc470 into master Oct 29, 2019
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.