Roadshow TCP/IP stack notes (2020-09-05) ======================================== 1. The demonstration version and the commercial release Roadshow is a commercial product which can be bought online or on CD-R from "http://www.apc-tcp.de". Support is available at "http://roadshow.apc-tcp.de". A free demonstration version as well as the free software development kit (SDK) are available on the same web site. The demonstration version is almost identical to the commercial version, but there is one important difference: the demonstration version will run only for a short period of time after which it stops. Once stopped, it will remain in memory, which means that you may have to restart your Amiga in order to give Roadshow another try. Because it remains in memory, it also means that you cannot easily shut it down and start a different TCP/IP stack. Sorry! This is not how it should work, but a different way to make the demonstration version work has not yet made itself obvious :-( For information on how to make the TCP/IP stack work again which you used before, after you have given Roadshow a try, please see section 7 of these notes (at the end of this document). 1.1. Upgrading the demonstration version to the commercial release Once you have installed the demonstration version, configured and tested it, you might want to buy the commercial release. You can retain your current Roadshow setup (keeping all the configuration files, etc.) and have those parts of Roadshow replaced with the versions which are only available as part of the commercial release. In order to perform this upgrade to the commercial release, just start the installation for the commercial Roadshow version. The installation script will detect if you currently have a demonstration version of Roadshow installed (all demonstration versions of Roadshow 1.9-1.14 are supported). If the demonstration version is found, its libraries and network device drivers will be upgraded to the current commercial release. The script will then exit. Please note that if you already have the commercial Roadshow version installed, then the installation script will prompt you to choose between making a backup copy of your configuration files and of uninstalling the Roadshow software. In that case, you might want to choose neither option and abort the installation instead. 2. Preparations for installing Roadshow Before you begin you might want to dip into the documentation provided in the "Documents" drawer. Specifically, the file "Documents/Installation.guide" should be useful when you are setting up your network for the first time. Printable versions of the documentation are available in the "Documents/Printable" drawer. The files in this drawer are in PDF format, suitable for printing through tools such as APDF or any other program which can process and output this document format. Note that the big Roadshow documentation is more than 100 pages in size and may be best used for reference. 3. The installation script Note that if you want to try Roadshow on MorphOS, then the installation script may not work for you. Sorry, this is how it is at the moment, and I do not have any good idea on how to fix this. You can, however, install the single components of Roadshow manually by copying the contents of the "Workbench" drawer to your system partition. Careful: do not copy the "Workbench/S/User-Startup" file! You might want to add the contents of the "Workbench/S/User-Startup" to your "S:User-Startup" file. To begin the installation, double-click on the "Install_Roadshow" icon. The installation script gives you a choice to run in "advanced" or "expert" mode. Picking "expert" allows for a few more choices to be made than the "advanced" mode. For example, in "expert" mode you will always be asked if you want to install a special version of the software, optimized for Amigas with MC68020 CPUs (or better), regardless of whether your current Amiga computer supports this (note: only the commercial version includes these specially optimized versions of the software). Note that if the installation script detects that Roadshow may be already installed, it will ask you if you want to uninstall Roadshow first. The installation script can run in "pretend mode", meaning that if you choose not install for real, then a log of all installation operations will be generated instead, for you to review. Lastly, the installation script can be rerun after you have installed the software and will give you two new choices: 1) you can choose to uninstall the Roadshow software previously installed, and 2) you can have it make backup copies of your current Roadshow configuration files. 4. Installing Roadshow under Amiga operating system version 2 The minimum installation requirements for Roadshow are an Amiga with operating system version 2.04 and 2 MBytes of main memory. But the tools provided for reading the documentation may actually require operating system version 3.0 to work out properly. This means that you may have to use your preferred document reader or text editor to open and review the documentation before you can install Roadshow on your system. The installation procedure for Amiga operating system version 2 should be identical to the procedure used for Amiga operating system version 3 and beyond, though. If need be, you can also install Roadshow manually by copying the necessary files to your system partition. 5. Uninstalling Roadshow Both the demonstration and commercial releases of Roadshow can be uninstalled by restarting the installation script. It will detect if you currently have Roadshow installed and then prompt you to choose between making a backup of your current Roadshow configuration files and uninstalling Roadshow. To uninstall Roadshow, choose the latter. The script will then proceed to remove all the files which the installation script copied to your system. When the script is finished, you should reboot your system as soon as convenient. Please note that if you do not currently have Roadshow installed on your system, then the installation script will proceed assuming that you want to install Roadshow. 6. The software development kit The software development kit (not included with the demonstration version: visit the web site to download it) is provided in the "SDK" drawer, which does not have an icon to double-click on. This drawer contains documentation, source code, examples and other files helpful and/or required if you want to create new software for Roadshow. 7. Creating a stripped-down installation floppy disk You might want to create a stripped-down version of the installation disk, for use with installing the TCP/IP stack from a single floppy disk. Here is what you can do: - You need to retain the contents of the "Workbench" drawer, the "Installer" program and the "Install_Roadshow" installation script. - From the "Workbench" drawer, you will need to remove a few files which are not strictly essential to the operation of the TCP/IP stack and the installation script. - The following shell commands can be removed to save space: Workbench/C/ftp Workbench/C/tcpdump Workbench/C/wget - If you do not need the PPP/PPPoE drivers to be installed, the following drawers can be removed: Workbench/Devs/Networks Workbench/S/PPP-Configurations And you can also remove the following shell commands: Workbench/C/ppp_dialer Workbench/C/ppp_connector Workbench/C/ppp_sample - If you know that you are not going to need the special MC68020-optimized version of bsdsocket.library, then you can remove the following drawer and its content: Workbench/Libs/020 Alternatively, if you know that the special MC68020-optimized version is what you need, you can replace the plain MC68000/MC68010 version with the MC68020-optimized version and then delete the plain version: Copy Workbench/Libs/020/bsdsocket.library Workbench/Libs Delete Workbench/Libs/020 ALL QUIET Note that you can, of course, always copy the relevant files manually back to your system partition later. It's just that the installation script will adapt to the missing data and proceed safely if you take care in removing the files and drawers described above. Removing all of the files and drawers described above should make enough room for the remaining installation files to fit onto a single 880K floppy disk. 8. Going back to Miami/AmiTCP/etc. after you have tried Roadshow Should you find that Roadshow does not compare well to the TCP/IP stack you were using before, and you need to return it, please note that you must uninstall Roadshow first. Otherwise, the TCP/IP stack you were using before will probably not run correctly any more. This is because the Roadshow installation script places the file "bsdsocket.library" in your LIBS: folder, and this prevents other TCP/IP stacks such as "Miami" or "AmiTCP" from working correctly. So, in order to make "Miami" or "AmiTCP" work again, you should either rerun the Roadshow installation script (which includes an option to remove the files it installed before), or delete the file "LIBS:bsdsocket.library" manually instead. 9. Changes Roadshow 1.14 (23.08.2020) - The "bsdsocket.library" version text now includes information which allows you to quickly verify whether you have the version for all Amigas installed, or the version which requires at least an 68020 CPU. Also included is the Roadshow release version which the "bsdsocket.library" is a part of. In this case, it would be "Roadshow 1.14". To inspect this information you will need to use the following shell command: Version full file bsdsocket.library Please note that this version information will not be printed if you use the "Version bsdsocket.library" command instead. - The vsyslog() function in bsdsocket.library did not remove the facility information from the "priority" parameter and ended up truncating the severity information instead, which it should have left unchanged. Thanks go to Patrik Axelsson who reported the problem and provided a test case! - The WaitSelect() function in bsdsocket.library now waits exactly as long as the provided timeout requires instead of rounding the time according to what the 1994 4.4BSD-Lite2 kernel required. This both simplified operations and avoids possible overflow errors which might result from the seconds/microseconds conversion. - If the WaitSelect() function in bsdsocket.library is called with a zero length timeout (polling), it will update any user-supplied signal conditions and also check if the interrupt signal had been set. In previous library versions the results of this update operation did not match what waiting for a timeout would have produced, triggering side-effects such as programs stopping as if an interrupt signal had been sent, with no such signal having been sent. - Whenever the TCP/IP stack needs to learn about the current system time it will no longer update a global variable and refer to it later. Instead, the current system time will be queried and stored in a local variable. This is a more robust approach since it avoids side-effects from concurrent Processes and the TCP/IP stack itself updating the time. - The "DEVS:Internet/name_resolution" configuration file now supports a new option which can be either "prefer static" or "prefer dynamic". The default is "prefer dynamic". This option selects whether or not dynamically-added DNS resolver addresses will be used first ("dynamic") or if the DNS resolver addresses in the "DEVS:Internet/name_resolution" configuration file will be used first. If the DHCP server provided your Amiga with a DNS server which is unresponsive or unacceptable, you can now override it. - The "DEVS:Internet/servers" configuration file now supports a new option "NOREQ/S" which can be used to make any shell command executed avoid showing AmigaDOS requester windows, such as prompting for a disk to be inserted. This should aid in making automation a bit smoother. - Roadshow version 1.14 ships with an "rsh" command which in the absence of a more robust "ssh" command could be useful to control other computers from your Amiga. Roadshow version 1.14 also has a built-in remote shell server itself, which is disabled by default. In order to enable it, you would need to add the following line to the "DEVS:Internet/servers" file: shell stream internal The "shell" service in the line above is the remote shell service on TCP port 514 which was first introduced way back in 1977 and is now considered obsolete and outrageously insecure. It may still be useful on the Amiga, bringing Roadshow closer to functional parity with the 1992 "Envoy" product which, too, allowed for remote code execution and multiprocessing. Because no security features are implied or enforced by the built-in Roadshow "shell" service, you should only enable it if you can guarantee that it will not be misused. The built-in Roadshow "shell" service enables you to combine several Amiga shell commands into a single line. These commands must be separated by ";" characters, like you would do with a Unix shell. The "shell" service will rewrite the command string, replacing each ";" character with a line feed, suitable for use with the AmigaDOS shell. This rewrite operation respects the syntax used by the AmigaDOS shell, only replacing ";" characters which would otherwise be recognized as introducing a comment text. Please note that use of ";" may require some tricky escaping/quoting on the "rsh" command line. - Fixed a long-standing bug (since 2009) in the NetShutdown command which prevented the shutdown timeout & recall procedure from working. Thanks go to Timm S. Mueller for finding and reporting it! - The "rsh" client command has been updated to produce better error error messages, providing more detail as to what went wrong. Errors caused by timeouts now show the timeout being exceeded as the problem. - If the "rsh" client receives an error message from the remote server, it will now exit with an error status set after having printed the message. Previous versions only printed the message but failed to exit, as they should have done. - The "rsh" client command could corrupt memory while reading and printing the remote server's output. Thanks go to Patrik Axelsson for finding this problem! - The "ftp" client needs at least 12000 bytes of stack space to work properly without crashing. It now uses a minimum default stack size of 20000 bytes, to be on the safe side. Thanks go to Patrik Axelsson for finding this problem! - Rebuilt the "ftp" client with the updated clib2 libunix.a which includes a fix for the __translate_unix_to_amiga_path_name() function to the effect that '//' are now correctly dealt with. - Rebuilt the "tcpdump" command with the updated clib2 libunix.a which includes a fix for the __translate_unix_to_amiga_path_name() function to the effect that '//' are now correctly dealt with. - The "NetLogViewer" program has been updated to render its message list more quickly and to avoid leaving rendering errors in the display. The program no longer requires Kickstart 3.0 or better to work, finally making it compatible with Kickstart 2.04. This in turn means that Roadshow is now fully Kickstart 2.04 compatible in all its parts. Thanks go to Patrik Axelsson for showing me that the rendering needed work, as well as the small bugs still present after all those years! - The "AddNetInterface" command now makes sure that it has more than 4096 bytes of stack space available. It requires a minimum of 8000 bytes to be on the safe side. This is to address stack and memory corruption which seems to have been a problem, and a hard to detect one, too, for a very long time. Thanks go to Patrik Axelsson who reported the problem! - Set the default DHCP timeout used by the "AddNetInterface" command to 10 seconds. This used to be 30 seconds, which could make your system startup stall for much too long if your Amiga had not been connected to the local network or the local DHCP server was not listening. Ten seconds should still be more than enough, and you can still override the default, of course. - Set the default DHCP timeout used by the "ConfigureNetInterface" command to 10 seconds so that it is consistent with what the AddNetInterface command uses. - The "wget" command now supports the "--timeout" options, and in particular the "--connect-timeout" option, which required a replacement for the connection operation. Please note that the connection timeout is limited to a maximum of 75 seconds due to limitations in how far the TCP/IP stack allows a connection attempt to linger. You can specify longer connection timeouts, but in practice the TCP/IP stack will not wait longer than 75 seconds. Thanks go to Patrik Axelsson who pointed out the problem! - The "wget" command's "--read-timeout" option now works as it should for the first time. Note that the "--timeout" option still has no effect on the DNS lookup (neither has the "--dns-timeout" option) because it depends upon the Unix alarm/signal mechanism which does not exist in this form on the Amiga. - The "wget" command could fail to print the correct error message if an error occurred trying to establish a connection. Thanks go to Patrik Axelsson who pointed out the problem! - Rebuilt the "wget" command with the updated clib2 libunix.a which includes a fix for the __translate_unix_to_amiga_path_name() function to the effect that '//' are now correctly dealt with. Roadshow 1.13 (24.4.2017) - Tomasz Potrykus provided Polish localization files for Roadshow, "bsdsocket.library", "usergroup.library", "ppp-ethernet.device" and "ppp-serial.device". Thank you very much! - The "ping" command sports a new switch which controls how many packets should be sent ("-c"). This parameter now also implies a test to see for how many packets sent there were matching responses. If fewer responses were received than there were packet sent, "ping" will now exit with return code 5, which can be tested for in shell scripts with "if warn .. endif". Further switches have been added which control how long packets should be sent before the program exits ("-t" and "-o"). - Fixed several bugs in "bsdsocket.library", the "SampleNetSpeed" and "ppp_sample" programs all related to the same problem: combining GetMsg() and WaitIO() is unsound and may lead to memory corruption and seemingly random crashes. You either should use GetMsg() or WaitIO(), but not both in the same context. Thanks go to Sebastian Bauer for finding the problem and convincing my stubborn self to take this issue more seriously! - The "ConfigureNetInterface" command tested the wrong parameter when checking the DHCPUNICAST/K parameter. Fixed. - The 68020+ builds of "bsdsocket.library", "ppp-ethernet.device" and "ppp-serial.device" no longer crash when opened on plain 68000 or 68010 systems. They now just fail to open at all. I added these tests because of problems with the Roadshow installation script which were caused by the CPU type test to mistakenly offer the option to install the 68020+ builds on plain 68000 or 68010 systems. - Updated the Roadshow installation script in order to make the test for the CPU type more robust. It turned out that the "(patmatch ..)" function in versions of the "Installer" tool prior to V43 does not work as documented. In fact, that function barely works at all. This problem broke the old installation script. The new installation script now tests if the CPU type is "68000" or "68010" in a simpler fashion. - The "bsdsocket.h" and "usergroup.h" inline header files for use with GCC 68k have been rebuilt using fd2pragma 1.264 because the versions built with fd2pragma 1.171 were reported not to work correctly. These rebuilt versions are now part of the SDK. Thanks go to Anders Ali Lindgren for reporting the issue! - The "TCP:" handler leaked memory whenever it was opened, 144 bytes to be precise. Thanks go to Patrik Axelsson who detected the problem and provided a test case! - Data received by an open connection and which is never read can accumulate a lot of memory over time. While the TCP/IP stack will eventually signal to the remote that the data has not been read by the client yet, so much memory may have been allocated to hold it that your system runs out of free memory first. Due to performance considerations, and also because it made the data reception from the network interface simpler, Roadshow always allocates a "cluster" of 2048 bytes for each inbound packet, and the received data remains in that "cluster" until it has been processed, even if only 40 bytes of memory would have been sufficient to keep it, for example. This can be problematic in the long run when 20 Megabytes of memory have piled up which contain only some 32 Kilobytes of unread data. To address this quirk, new options have been added to the "RoadshowControl" command. "RoadshowControl set if.receive.useclusters = 0" will make Roadshow store incoming data using as little free memory as possible, at the expense of slightly higher processing effort. In this mode the TCP/IP stack should run much more stable in the long term. Again, Patrik Axelsson found the problem, provided a test case and tested the updated bsdsocket.library version. Thank you very much! - Both the "ping" and "traceroute" commands show greater accuracy when measuring the lengths of time intervals. As Patrik Axelsson demonstrated, the measurements taken could be very coarse on the Amiga 2000, using timer.device version 39. The new measurement method is the most accurate which the Amiga operating system permits. Again, thanks go to Patrik for finding the problem and for providing a test case! - The wget command no longer gets itself into trouble when too little shell stack space is available. Also, the transmission rate calculation finally works as intended. The problem was with how the length of time intervals was calculated, which assumed that the number of microseconds in each time stamp was stored as a signed number. The processing of the short form command options (e.g. "-q" for "--quiet") never worked correctly either, with a bug in clib2 causing the problem. Both the stack space and the short form command option problems were reported by Patrik Axelsson. Thank you very much! ping 4.13 (3.5.2017) - The main ping loop could wind up checking the state of the timeout request, even though it had never been used. This had repercussions because it was then followed by a WaitIO() call on the timerequest, which implies a Remove(), triggering an Enforcer hit. Thanks go to Patrik Axelsson for spotting the problem! - The average round-trip time printed is no longer rounded to microsecond precision. The rounding had curious side-effects when printing the average round-trip time for a single packet, which did not equal the minimum and maximum round trip times. - All trip times are now printed with a fixed number of digits (three) following the decimal point. Also, the trip time is always printed as a series of digits and the decimal point, never in scientific notation (mantissa with exponent). - An invalid "-t" option value is no longer ignored, but results in an error message being printed and the ping command exiting with a failure code set. Roadshow 1.12 (5.10.2016) - Added new API functions for use with name resolution in order to make it easier to port application software. The new functions include the BSD style gethostbyname_r() and gethostbyaddr_r(), as well as versions of getaddrinfo(), gai_strerror(), freeaddrinfo() and getnameinfo() which are currently restricted to IPv4 addresses. - Processes which share the same library base can now safely perform overlapping name resolution operations, without danger of corrupting each other's query results. Please note that the gethostbyname_r() and gethostbyaddr_r() functions need to be used to be completely safe from accidental corruption. If your code still uses gethostbyname() or gethostbyaddr() then the recent changes will make the name resolution much more robust, but because the functions share state information which is available only once (as part of the library base) conflicts are still possible. - Reworked the ftp client command, improving robustness. This means that buffer overrun issues have been corrected which had remained unfixed since 1994. - The AddNetInterface and ConfigureNetInterface commands, as well as bsdsocket.library itself, have been enhanced to allow for the DHCP server to send address offers using unicast datagrams. Previously, only broadcast datagrams were supported. The unicast support brought a small bug to the surface to the effect that bsdsocket.library never asked the DHCP server to respond with a broadcast datagram. This bug has been fixed. - The bsdsocket.library API function QueryInterfaceTags() tag 'IFQ_HardwareAddressSize' would copy more than 16 bytes for the hardware address. In fact, it would copy as many bytes as there were bits in the address (usually, that would be 48 bytes). This bug could cause the ShowNetStatus command to crash and has been fixed in bsdsocket.library itself. - By request a new ready-made network interface configuration file has been added, for use with the "cnet16.device" driver. This file is found in the "Storage/NetInterfaces" drawer and is called "CNet16". - The TCP/IP stack's internal timekeeping relies upon the system time (as counted in microseconds) to increase "monotonously". This means that as time passes, the system microsecond counter only ever increases and never decreases, and (hopefully) never stays the same. However, this was never guaranteed because you could manually change the clock time, or a tool could automatically switch between daylight saving time and back. These clock changes could cause Roadshow to drop name resolution and routing entries prematurely, and it could also trip up the TCP protocol's time measurements. bsdsocket.library has been updated so that it is no longer sensitive to system clock time changes. Changing the system clock time can no longer result in cache entries getting dropped, or the TCP protocol's time measurements to slip. Ingo Schmitz and Harald Meinzer kindly tested the 68000 version of the updated bsdsocket.library (version 4.321). Thank you very much! Roadshow 1.11 (9.11.2013) - Improved the robustness of the ppp-serial.device and ppp-ethernet.device drivers on Amiga computers with 68000 and 68010 CPUs. Thanks go to Christian Vogelgsang for raising the issue and testing the changes. - The version of ppp-serial.device intended for all Amigas no longer allows Van-Jacobson header compression when running on an Amiga with a 68000 or 68010 CPU. This restriction may be lifted in a future version of the driver. - The ppp-ethernet.device no longer supports Van-Jacobson header compression, because that feature is not supported by the PPPoE protocol anyway. - Replaced the copying routines in the bsdsocket.library version intended for all Amigas, specifically those with 68000 and 68010 CPUs. This change was prompted by the investigation which led to the robustness improvements for the ppp-serial.device and ppp-ethernet.device drivers. The version of bsdsocket.library for Amigas with 68020 CPUs (or better) is essentially the same as before, with its version number bumped to match the library version for all Amigas. - The documentation no longer recommends tweaking the tcp.do_timestamps and tcp.do_win_scale settings. Roadshow 1.10 (19.10.2013) - Added a ready-made, new network interface configuration file for the "X-Surf 100" Ethernet card. - The "ftp" command did not process the "host" parameter correctly, and the optional "port" parameter was ignored. Thanks go to Alexandre Balaban for finding the issue. - The "ppp_connector", "ppp_dialer" and "ppp_sample" commands have been replaced with the correct versions, suitable for "classic" 68000-based Amigas. Previously, these commands were built to run only on AmigaOS 4; I had accidentally copied them from the wrong distribution drawer :-( - The optimized IP/TCP/UDP checksum calculation and verification code in the bsdsocket.library version intended for all Amigas (with 68000 and 68010 CPUs, too, such as the A500, A2000 and A600) was not stable and had to be replaced by the portable version. This change fixes crashes on these Amiga types. Amigas with 68020/030/040/060 were unaffected by these problems. Thanks go to Thorham for verifying the fixes. Roadshow 1.9 (24.2.2013): - Network interface names cannot be longer than 15 characters. Shame on me, I did not remember this when I added the new MorphOS interface files. Network interface file names have been changed as follows: Freescale-MPC52xx -> MPC52xx Marvell-Discovery-II -> Discovery-II Poseidon-Asix -> Asix Poseidon-Davicom -> Davicom Poseidon-MosChip -> MosChip Poseidon-Pegasus -> Pegasus Realtek-RTL8139 -> RTL8139 - USB device driver names have been corrected for the Asix, Davicom and MosChip network interface configuration files. - The documentation has been updated to explain in greater detail how the "DEVS:Internet/servers" configuration file looks like, and how one may convert "inetd.conf" configuration data so that it matches the format of "DEVS:Internet/servers". - Added ready-made configuration lines to the "DEVS:Internet/servers" files for use with the Samba server suite. These lines have been commented out and can be easily reactivated after you have installed Samba on your Amiga (take care, installing Samba is a challenging undertaking!). Roadshow 1.8 (19.1.2013): - Added more example network interface files for use with MorphOS: "Freescale-MPC52xx", "Marvell-Discovery-II" and "Realtek-RTL8139". Roadshow 1.7 (19.1.2013): - Updated the example network interface files "SunGEM" and "VIA-Rhine" to include the line 'state=online', which switches the respective network device drivers online even if DHCP configuration is not used. Roadshow 1.6 (10.1.2013): - Modified AddNetInterface (4.52) to switch an interface online prior to starting DHCP negotiation. This improves compatibility with the MorphOS "sungem_eth.device" and "via_rhine_pci.device" drivers. - AddNetInterface now supports "up", "down", "online" and "offline" as parameters for the "state" configuration file option. - Updated AddNetInterface source code is now part of the SDK. - Added example network interface files "SunGEM" and "VIA-Rhine" for MorphOS.