Pymodbus 2.2.0 (#375)

* #357 Support registration of custom requests

* #368 Fixes write to broadcast address

When writing to broadcast address (unit_id=0) there should be no response according to the Modbus spec. This fix changes expected_response_length to 0 when writing to unit_id=0. This will break any existing code that is improperly using unit_id 0 for a slave address.

* Bump version to 2.2.0

Fix #366 Update failures in sql context

Update Changelog

Fix major minor version in example codes

* Fix #371 pymodbus repl on python3

* 1. Fix tornado async serial client `TypeError` while processing incoming packet.
2. Fix asyncio examples.
3. Minor update in factory.py, now server logs prints received request instead of only function cod

* [fix v3] poprawa sprawdzania timeout

* Release candidate for pymodbus 2.2.0

*  Fix #377 when invalid port is supplied and minor updates in logging

* #368 adds broadcast support for sync client and server

Adds broadcast_enable parameter to client and server, default value is False. When true it will treat unit_id 0 as broadcast and execute requests on all server slave contexts and not send a response and on the client side will send the request and not try to receive a response.

* #368 Fixes minor bug in broadcast support code

* Fixed erronous CRC handling

If the CRC recieved is not correct in my case my slave got caught in a deadlock, not taking any new requests. This addition fixed that.

* Update Changelog

* Fix test coverage

* Fix #387 Transactions failing on 2.2.0rc2.

* Task Cancellation and CRC Errors

Alternate solution for #356 and #360.

Changes the RTU to make the transaction ID as the unit ID instead of an ever incrementing number.

Previously this transaction ID was always 0 on the receiving end but was the unique transaction ID on sending.

As such the FIFO buffer made the most sense. By tying it to the unit ID, we can recover from failure modes such as: -
- Asyncio task cancellations (eg. timeouts) #360
- Skipped responses from slaves. (hangs on master #360)
- CRC Errors #356
- Busy response

* Cherry pick commit from PR #367 , Update changelog , bump version to 2.2.0rc4

* #389 Support passing all serial port parameters to asynchronous server

* Fix BinaryPayloadDecoder and Builder wrt to coils

* Misc updates, bump version to 2.2.0

* ReportSlaveIdResponse now tries to get slave id based on server identity for pymodbus servers

* Update missing bcrypt requirement for testing

.coveragerc

@@ -1,3 +1,4 @@
 [run]
 omit =
-    pymodbus/repl/*
+    pymodbus/repl/*
+    pymodbus/internal/*

CHANGELOG.rst

@@ -1,6 +1,38 @@
-Version 2.1.1
+Version 2.2.0
 -----------------------------------------------------------
+**NOTE: Supports python 3.7, async client is now moved to pymodbus/client/asychronous**
+```
+from pymodbus.client.asynchronous import ModbusTcpClient
+```
+
+* Support Python 3.7
+* Fix to task cancellations and CRC errors for async serial clients.
+* Fix passing serial settings to asynchronous serial server.
+* Fix `AttributeError` when setting `interCharTimeout` for serial clients.
 * Provide an option to disable inter char timeouts with Modbus RTU.
+* Add support to register custom requests in clients and server instances.
+* Fix read timeout calculation in ModbusTCP.
+* Fix SQLDbcontext always returning InvalidAddress error.
+* Fix SQLDbcontext update failure
+* Fix Binary payload example for endianess.
+* Fix BinaryPayloadDecoder.to_coils and BinaryPayloadBuilder.fromCoils methods.
+* Fix tornado async serial client `TypeError` while processing incoming packet.
+* Fix erroneous CRC handling in Modbus RTU framer.
+* Support broadcasting in Modbus Client and Servers (sync).
+* Fix asyncio examples.
+* Improved logging in Modbus Server .
+* ReportSlaveIdRequest would fetch information from Device identity instead of hardcoded `Pymodbus`.
+* Fix regression introduced in 2.2.0rc2 (Modbus sync client transaction failing)
+* Minor update in factory.py, now server logs prints received request instead of only function code
+
+```
+# Now
+DEBUG:pymodbus.factory:Factory Request[ReadInputRegistersRequest: 4]
+# Before
+DEBUG:pymodbus.factory:Factory Request[4]
+
+```
+
 
 Version 2.1.0
 -----------------------------------------------------------

examples/common/async_asyncio_serial_client.py

@@ -49,9 +49,6 @@ async def start_async_test(client):
     # which defaults to `0x00`
     # ----------------------------------------------------------------------- #
     try:
-        log.debug("Reading Coils")
-        rr = client.read_coils(1, 1, unit=UNIT)
-
     # ----------------------------------------------------------------------- #
         # example requests
     # ----------------------------------------------------------------------- #
@@ -137,7 +134,7 @@ async def start_async_test(client):
     # socat -d -d PTY,link=/tmp/ptyp0,raw,echo=0,ispeed=9600 PTY,
     # link=/tmp/ttyp0,raw,echo=0,ospeed=9600
     loop, client = ModbusClient(schedulers.ASYNC_IO, port='/tmp/ptyp0',
-                                baudrate=9600, timeout=2, method="rtu")
+                                baudrate=9600, method="rtu")
     loop.run_until_complete(start_async_test(client.protocol))
     loop.close()
 

examples/common/async_tornado_client_serial.py

@@ -24,7 +24,10 @@
 # configure the client logging
 # ---------------------------------------------------------------------------#
 import logging
-logging.basicConfig()
+
+FORMAT = ('%(asctime)-15s %(threadName)-15s'
+          ' %(levelname)-8s %(module)-15s:%(lineno)-8s %(message)s')
+logging.basicConfig(format=FORMAT)
 log = logging.getLogger()
 log.setLevel(logging.DEBUG)
 

examples/common/asynchronous_server.py

@@ -20,6 +20,7 @@
 from pymodbus.transaction import (ModbusRtuFramer,
                                   ModbusAsciiFramer,
                                   ModbusBinaryFramer)
+from custom_message import CustomModbusRequest
 
 # --------------------------------------------------------------------------- # 
 # configure the service logging
@@ -92,6 +93,8 @@ def run_async_server():
         co=ModbusSequentialDataBlock(0, [17]*100),
         hr=ModbusSequentialDataBlock(0, [17]*100),
         ir=ModbusSequentialDataBlock(0, [17]*100))
+    store.register(CustomModbusRequest.function_code, 'cm',
+                   ModbusSequentialDataBlock(0, [17] * 100))
     context = ModbusServerContext(slaves=store, single=True)
 
     # ----------------------------------------------------------------------- # 
@@ -105,15 +108,16 @@ def run_async_server():
     identity.VendorUrl = 'http://github.com/bashwork/pymodbus/'
     identity.ProductName = 'Pymodbus Server'
     identity.ModelName = 'Pymodbus Server'
-    identity.MajorMinorRevision = '1.5'
+    identity.MajorMinorRevision = '2.2.0'
 
     # ----------------------------------------------------------------------- # 
     # run the server you want
     # ----------------------------------------------------------------------- # 
 
     # TCP Server
 
-    StartTcpServer(context, identity=identity, address=("localhost", 5020))
+    StartTcpServer(context, identity=identity, address=("localhost", 5020),
+                   custom_functions=[CustomModbusRequest])
 
     # TCP Server with deferred reactor run
 

examples/common/callback_server.py

@@ -132,7 +132,7 @@ def run_callback_server():
     identity.VendorUrl = 'http://github.com/bashwork/pymodbus/'
     identity.ProductName = 'pymodbus Server'
     identity.ModelName = 'pymodbus Server'
-    identity.MajorMinorRevision = '1.0'
+    identity.MajorMinorRevision = '2.2.0'
 
     # ----------------------------------------------------------------------- #
     # run the server you want

examples/common/custom_datablock.py

@@ -68,7 +68,7 @@ def run_custom_db_server():
     identity.VendorUrl = 'http://github.com/bashwork/pymodbus/'
     identity.ProductName = 'pymodbus Server'
     identity.ModelName = 'pymodbus Server'
-    identity.MajorMinorRevision = '1.0'
+    identity.MajorMinorRevision = '2.2.0'
 
     # ----------------------------------------------------------------------- #
     # run the server you want

examples/common/custom_message.py

@@ -20,6 +20,7 @@
 from pymodbus.pdu import ModbusRequest, ModbusResponse, ModbusExceptions
 from pymodbus.client.sync import ModbusTcpClient as ModbusClient
 from pymodbus.bit_read_message import ReadCoilsRequest
+from pymodbus.compat import int2byte, byte2int
 # --------------------------------------------------------------------------- #
 # configure the client logging
 # --------------------------------------------------------------------------- #
@@ -40,15 +41,41 @@
 
 
 class CustomModbusResponse(ModbusResponse):
-    pass
+    function_code = 55
+    _rtu_byte_count_pos = 2
+
+    def __init__(self, values=None, **kwargs):
+        ModbusResponse.__init__(self, **kwargs)
+        self.values = values or []
+
+    def encode(self):
+        """ Encodes response pdu
+
+        :returns: The encoded packet message
+        """
+        result = int2byte(len(self.values) * 2)
+        for register in self.values:
+            result += struct.pack('>H', register)
+        return result
+
+    def decode(self, data):
+        """ Decodes response pdu
+
+        :param data: The packet data to decode
+        """
+        byte_count = byte2int(data[0])
+        self.values = []
+        for i in range(1, byte_count + 1, 2):
+            self.values.append(struct.unpack('>H', data[i:i + 2])[0])
 
 
 class CustomModbusRequest(ModbusRequest):
 
-    function_code = 1
+    function_code = 55
+    _rtu_frame_size = 8
 
-    def __init__(self, address):
-        ModbusRequest.__init__(self)
+    def __init__(self, address=None, **kwargs):
+        ModbusRequest.__init__(self, **kwargs)
         self.address = address
         self.count = 16
 
@@ -74,12 +101,12 @@ def execute(self, context):
 
 class Read16CoilsRequest(ReadCoilsRequest):
 
-    def __init__(self, address):
+    def __init__(self, address, **kwargs):
         """ Initializes a new instance
 
         :param address: The address to start reading from
         """
-        ReadCoilsRequest.__init__(self, address, 16)
+        ReadCoilsRequest.__init__(self, address, 16, **kwargs)
 
 # --------------------------------------------------------------------------- #
 # execute the request with your client
@@ -90,7 +117,8 @@ def __init__(self, address):
 
 
 if __name__ == "__main__":
-    with ModbusClient('127.0.0.1') as client:
-        request = CustomModbusRequest(0)
+    with ModbusClient(host='localhost', port=5020) as client:
+        client.register(CustomModbusResponse)
+        request = CustomModbusRequest(1, unit=1)
         result = client.execute(request)
-        print(result)
+        print(result.values)

examples/common/custom_message_async_clients.py

@@ -0,0 +1,124 @@
+#!/usr/bin/env python
+"""
+Pymodbus Synchronous Client Examples
+--------------------------------------------------------------------------
+
+The following is an example of how to use the synchronous modbus client
+implementation from pymodbus.
+
+It should be noted that the client can also be used with
+the guard construct that is available in python 2.5 and up::
+
+    with ModbusClient('127.0.0.1') as client:
+        result = client.read_coils(1,10)
+        print result
+"""
+import struct
+# --------------------------------------------------------------------------- #
+# import the various server implementations
+# --------------------------------------------------------------------------- #
+from pymodbus.pdu import ModbusRequest, ModbusResponse, ModbusExceptions
+from pymodbus.client.sync import ModbusTcpClient as ModbusClient
+from pymodbus.bit_read_message import ReadCoilsRequest
+from pymodbus.compat import int2byte, byte2int
+# --------------------------------------------------------------------------- #
+# configure the client logging
+# --------------------------------------------------------------------------- #
+import logging
+logging.basicConfig()
+log = logging.getLogger()
+log.setLevel(logging.DEBUG)
+
+# --------------------------------------------------------------------------- #
+# create your custom message
+# --------------------------------------------------------------------------- #
+# The following is simply a read coil request that always reads 16 coils.
+# Since the function code is already registered with the decoder factory,
+# this will be decoded as a read coil response. If you implement a new 
+# method that is not currently implemented, you must register the request
+# and response with a ClientDecoder factory.
+# --------------------------------------------------------------------------- #
+
+
+class CustomModbusResponse(ModbusResponse):
+    function_code = 55
+    _rtu_byte_count_pos = 2
+
+    def __init__(self, values=None, **kwargs):
+        ModbusResponse.__init__(self, **kwargs)
+        self.values = values or []
+
+    def encode(self):
+        """ Encodes response pdu
+
+        :returns: The encoded packet message
+        """
+        result = int2byte(len(self.values) * 2)
+        for register in self.values:
+            result += struct.pack('>H', register)
+        return result
+
+    def decode(self, data):
+        """ Decodes response pdu
+
+        :param data: The packet data to decode
+        """
+        byte_count = byte2int(data[0])
+        self.values = []
+        for i in range(1, byte_count + 1, 2):
+            self.values.append(struct.unpack('>H', data[i:i + 2])[0])
+
+
+class CustomModbusRequest(ModbusRequest):
+
+    function_code = 55
+    _rtu_frame_size = 8
+
+    def __init__(self, address=None, **kwargs):
+        ModbusRequest.__init__(self, **kwargs)
+        self.address = address
+        self.count = 16
+
+    def encode(self):
+        return struct.pack('>HH', self.address, self.count)
+
+    def decode(self, data):
+        self.address, self.count = struct.unpack('>HH', data)
+
+    def execute(self, context):
+        if not (1 <= self.count <= 0x7d0):
+            return self.doException(ModbusExceptions.IllegalValue)
+        if not context.validate(self.function_code, self.address, self.count):
+            return self.doException(ModbusExceptions.IllegalAddress)
+        values = context.getValues(self.function_code, self.address,
+                                   self.count)
+        return CustomModbusResponse(values)
+
+# --------------------------------------------------------------------------- #
+# This could also have been defined as
+# --------------------------------------------------------------------------- #
+
+
+class Read16CoilsRequest(ReadCoilsRequest):
+
+    def __init__(self, address, **kwargs):
+        """ Initializes a new instance
+
+        :param address: The address to start reading from
+        """
+        ReadCoilsRequest.__init__(self, address, 16, **kwargs)
+
+# --------------------------------------------------------------------------- #
+# execute the request with your client
+# --------------------------------------------------------------------------- #
+# using the with context, the client will automatically be connected
+# and closed when it leaves the current scope.
+# --------------------------------------------------------------------------- #
+
+
+if __name__ == "__main__":
+    with ModbusClient(host='localhost', port=5020) as client:
+        client.register(CustomModbusResponse)
+        request = CustomModbusRequest(1, unit=1)
+        result = client.execute(request)
+        print(result.values)