protocol, short for "teletype network",
was born. And extended a number of times, to add features as the era required.
moved to /tables/serial.webt
The serial port state is described in ioctl structure, termios. The structures are as follows:
asm-generic/termios.h asm-generic/termbits.h #define NCC 8 #define NCCS 19 #define NCCS 19
struct termio { struct termios { struct termios2 { unsigned short c_iflag; tcflag_t c_iflag; tcflag_t c_iflag; /* input mode flags */ unsigned short c_oflag; tcflag_t c_oflag; tcflag_t c_oflag; /* output mode flags */ unsigned short c_cflag; tcflag_t c_cflag; tcflag_t c_cflag; /* control mode flags */ unsigned short c_lflag; tcflag_t c_lflag; tcflag_t c_lflag; /* local mode flags */ unsigned char c_line; cc_t c_line; cc_t c_line; /* line discipline */ unsigned char c_cc[NCC]; cc_t c_cc[NCCS]; cc_t c_cc[NCCS]; /* control characters */ }; }; speed_t c_ispeed; /* input speed */ speed_t c_ospeed; /* output speed */ };
The termios structure is almost identical to termio, with exception of more indirect specification of data type and more control characters (19 instead of 8).
The termios2 structure is almost identical to termios, with added fields for integer-specified input and output data rate. (Few port controllers allow different speeds in each direction, this is uncommon. Usually both input and output data rate match.)
The c_cc structure is an array of control characters; position in array corresponds to a function, value corresponds to character
The bit flags for control lines are as follows:
bit TIOCM_[ref]
0x0001 LE in DSR line Line Enable
0x0002 DTR OUT DTR line
0x0004 RTS OUT RTS line
0x0008 ST secondary transmit, usually unused
0x0010 SR secondary receive, usually unused
0x0020 CTS in CTS line
0x0040 CAR,CD in DCD line
0x0080 RNG,RI in RI line
0x0100 DSR in DSR line
0x2000 OUT1 OUT? Unassigned Programmable Output 1, usually unused
0x4000 OUT2 OUT? Unassigned Programmable Output 2, usually unused
0x8000 LOOP out loopback usually unused or used only for testing
ioctls as follows:
CLOCAL:
termios data structure flags:[ref]
0x100f CBAUD MASK: baud rates 0x0000 B0 no speed - hang up 0x0001 B50 0x0002 B75 0x0003 B110 0x0004 B134 0x0005 B150 0x0006 B200 0x0007 B300 0x0008 B600 0x0009 B1200 0x000a B1800 0x000b B2400 0x000c B4800 0x000d B9600 0x000e B19200,EXTA on some systems, external-clock input A 0x000f B38400,EXTB on some systems, external-clock input B; 38400 baudrate often needs a specific kludge 0x1000 CBAUDEX mask for extended baudrates flag, for rates not defined in POSIX.1 0x1000 BOTHER other unspecified baud rate 0x1001 B57600 0x1002 B115200 0x1003 B230400 highest usual value; top for FTDI at 12 MHz? 0x1004 B460800 0x1005 B500000 0x1006 B576000 0x1007 B921600 0x1008 B1000000 0x1009 B1152000 0x100a B1500000 0x100b B2000000 top for FTDI at 48 MHz? 0x100c B2500000 0x100d B3000000 0x100e B3500000 0x100f B4000000 0x0030 CSIZE MASK: data bits 0x0000 CS5 5 bits 0x0010 CS6 6 bits 0x0020 CS7 7 bits 0x0030 CS8 8 bits 0x0040 CSTOPB stop bits - 1 when unset, 2 when set 0x0080 CREAD enable receiver 0x0100 PARENB parity enabled 0x0200 PARODD odd parity - even when unset, odd when set 0x0400 HUPCL lower modem control lines (hang up) after last process closes device 0x0800 CLOCAL ignore modem control lines
0x0001 IGNBRK ignore BREAK if set; if IGNBRK=BRKINT=0, BREAK is received as 0x00 byte, if PARMRK set BREAK received as 0xFF 0x00 0x00 0x0002 BRKINT if IGNBRK not set, BREAK flushes I/O queue and (if allowed) sends SIGINT to terminal 0x0004 IGNPAR ignore framing and parity errors 0x0008 PARMRK meaningful if IGNPAR=0,INPCK=1; if set, parity error byte 0xXX is received as 0xFF 0x00 0xXX 0x0010 INPCK enable input parity check 0x0020 ISTRIP strip off 8th bit, MSB 0x0040 INLCR translate 0x0A to 0x0D (LF to CR) 0x0080 IGNCR ignore 0x0D 0x0100 ICRNL translate 0x0A to 0x0D (CR to LF, if IGNCR not set) 0x0200 IUCLC translate uppercase to lowercase 0x0400 IXON output XON/XOFF flow control enable 0x0800 IXANY if set, any character restarts output (not just XON/START/^Q) 0x1000 IXOFF input XON/XOFF flow control enable 0x2000 IMAXBEL non-POSIX; ring bell if queue full; linux ignores, acts like always set 0x4000 IUTF8 non-POSIX; in linux, allows correct erasing of characters with utf8
0x0001 OPOST implementation-defined input processing 0x0002 OLCUC translate lowercase to uppercase 0x0004 ONLCR translate 0x0A to 0x0D (LF to CR) 0x0008 OCRNL translate 0x0D to 0x0A (CR to LF) 0x0010 ONOCR no output of 0x0D/CR on column 0 0x0020 ONLRET don't output 0x0D/CR 0x0040 OFILL send fill characters for delay (keep line busy instead of pausing) 0x0080 OFDEL not in linux; if set, fill char is 0x7F/DEL, otherwise fill char is 0x00 0x0100 NLDLY MASK: newline delay (for mechanical terminals) 0x0000 NL0 0x0100 NL1 0x0600 CRDLY MASK: carriage return delay (for mechanical terminals) 0x0000 CR0 0x0200 CR1 0x0400 CR2 0x0600 CR3 0x1800 TABDLY MASK: horizontal tab delay (for mechanical terminals) 0x0000 TAB0 0x0800 TAB1 0x1000 TAB2 0x1800 TAB3,XTABS expand tabs to spaces 0x2000 BSDLY MASK: backspace delay (for mechanical terminals, not implemented) 0x0000 BS0 0x2000 BS1 0x4000 VTDLY MASK: vertical tab delay 0x0000 VT0 0x4000 VT1 0x8000 FFDLY MASK: form feed delay 0x0000 FF0 0x8000 FF1
0x0001 ISIG if set: if INTR, QUIT, SUSP, DSUSP received, generate corresponding signal 0x0002 ICANON canonical mode 0x0004 XCASE if set if ICANON set, terminal uppercase-only (not in linux) 0x0008 ECHO echo input to output 0x0010 ECHOE if ICANON set, if set ERASE erases preceding input character, WERASE erases word 0x0020 ECHOK if ICANON set, if set KILL erases current line 0x0040 ECHONL if ICANON set, echo 0x0A/NL even if ECHO not set 0x0080 NOFLSH disable flushing input/output queues when generating signals for INT, QUIT, SUSP 0x0100 TOSTOP send SIGTTOU 0x0200 ECHOCTL (non-POSIX) if ECHO set, show control chars other than 0x09/TAB, 0x0A/NL, START, STOP as ^x (eg 0x08/DEL = ^H) 0x0400 ECHOPRT (non-POSIX) if ECHO set if ICANON set characters printed as they are erased 0x0800 ECHOKE (non-POSIX) if ICANON set, if set, KILL erases each character on line 0x1000 FLUSHO (non-POSIX, not in linux) output being flushed, triggered by typing DISCARD char 0x2000 PENDIN (non-POSIX, not in linux) all chars in input queue reprinted when char read 0x4000 IEXTEN implementation-defined input processing 0x8000 EXTPROC
Canonical mode - sends data line by line (on CR or LF); noncanonical mode - sends each byte immediately
Raw mode - noncanonical, no echo, no special processing:
pos char hex ^x POSIX Linux needed-flags
0 ETX VINTR 03 ^C ISIG sends SIGINT
1 FS VQUIT 1c ^\ ISIG sends SIGQUIT
2 DEL VERASE 7f ICANON also 0x08, BS, ^H
3 NAK VKILL 15 ^U ICANON also ^X; erases input since last EOF
4 EOT VEOF 04 ^D ICANON send line immediately without witing for EOL; if first char of the line, send EOF/end of file
5 VTIME 0 not-ICANON time in milliseconds for noncanonical read
6 VMIN 0 not-ICANON minimum number of characters for noncanonical read
7 VSWTCH 0 no no n/a SystemV shell switch
8 DC1 VSTART 11 ^Q IXON XON, start output after suspended by VSTOP
9 DC3 VSTOP 13 ^S IXON XOFF, stop output until VSTART typed
10 SUB VSUSP 1a ^Z ISIG send SIGTSTP signal
11 VEOL 0 ICANON end of line
12 DC2 VREPRINT 12 ^R no ICANON,IEXTEN reprint unread characters
13 SI VDISCARD 0f ^O no no IEXTEN (start/stop discarding pending input)
14 ETB VWERASE 17 ^W no ICANON,IEXTEN word erase
15 SYN VLNEXT 16 ^V no IEXTEN quotes next input, removes special meaning
16 VEOL2 0 no no ICANON (another end of line)
EM VDSUSP (19) ^Y no no ISIG, IEXTEN (sends SIGTSTP)
DC4 VSTATUS (14) ^T no no ISIG? (display status, send SIGINFO - not in linux)
default value (from strace):
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
x03 x1c x7f x15 x04 x00 x00 x00 x11 x13 x1a x00 x12 x0f x17 x16 x00 x00 x00
VTIME and VMIN control read() behavior:
VTIME VMIN
0 0 read() immediately returns, with zero length (no data) if no byte available
0 nonzero read() blocks until VMIN bytes available
nonzero 0 read() returns after VTIME, with or without data
nonzero nonzero read() returns after VTIME or after VMIN bytes available
The age of modems is mostly over, the stage was seized by computer networks. Ethernet, wifi, and many other technologies replaced serial lines. Except in communication with various devices - the serial, usually in the form of TTL-level UART (Arduino, ESP8266, various microcontrollers) or RS232 (various peripherals) or RS485 (industrial buses), is still a king there. Even many USB devices act as serial ports, whether as /dev/ttyACM* or /dev/ttyUSB*.
The problem was encountered even in the old times. The telnet protocol, short for "teletype network",
was born. And extended a number of times, to add features as the era required.
Telnet itself is good for "plain" sending of data streams back and forth, with some additional negotiation functions. However, serial ports have a number of "out of band" features, controlled by ioctls - everything from bitrate to modem control lines to sending a break.
But there are ways for encoding such out-of-band activity as in-band signaling.
For telnet itself, the signaling is based on a dedicated escape character, 0xFF, named IAC, "interpret as command". After it, one or two other bytes are sent with the command and a parameter.
(if 0xFF has to be send through as a character, send it twice; first time it gets interpreted as IAC, the next command 0xff then means "send 0xff through".)
definition:
Several codes are defined:
Options: [ref]
The 0x2C option for the telnet serial port operation includes subcommands:
TODO: examples of command sequences
The responses to commands do not have to arrive in order.
CAUTION: The commands are not particularly synchronized with the data output (due to data buffering between the running program and the port output), so eg. commands to handle control lines can be executed before the nearly preceding data bytes made their way out of the port to freedom (or enslavement in another hardware).
For clarity, sequences "ff fa" and "ff f0" are written together as "fffa" and "fff0".
identify terminal type:
stop a process in terminal
Sometimes it is needed to make a remote serial port appear as local one, typically with TCP/IP network in between. Telnet-based serial connection can be utilized.
The same control sequences defined in original telnet are extended with the 0x2C (44d) code for the serial port related operations.
Here we need two ends of communication:
component for userspace implementation of character devices
https://github.com/pyserial/pyserial/blob/master/examples/rfc2217_server.py
Python has a highly powerful library for handling serial ports, aptly named pyserial.
rfc2217_server.py is a reference implementation of such a server.
Very handy for debugging, can be easily edited to serve other purposes.
Fairly complete support of pretty much all the baudrate and control line commands.
Runs on any platform on which Python 3 can be installed.
python3 ./rfc2217_server.py -v -v -v -v -v -p 5000 /dev/ttyUSB0(runs daemon on port 5000, physical port /dev/ttyUSB0, verbose about what it is doing)
Written in C. Available as a standard package for many distros (incl. Raspbian, OpenWRT). Runs as a daemon.
TODO: find out how good command support it has, if it can handle arbitrary baudrates.
ser2net(configuration in /etc/ser2net.conf)
http://lpccomp.bc.ca/remserial/
Does not seem to correctly handle the RFC2217 commands. Good enough for basic shoveling data back and forth without
need to handle control lines or change speeds.
Fairly simple C implementation but no support for RFC2217 commands.
https://gitlab.com/lars-thrane-as/ttynvt
Written in C. Userspace-implemented character device driver, via CUSE. Capable of handling ioctls
native-like.
Runs in a master process and two slave threads that handle the communication itself. When using strace to debug what's happening, don't forget to attach it to both the master process PID and two other (usually subsequent) PIDs or a lot of activity will appear to be missing.
Sends debug information to syslog. Copious data can be obtained by setting syslog level properly.
ttynvt -D 9 -M 199 -m 6 -n ttyNVT0 -S 192.168.1.10:5000(runs daemon that creates /dev/ttyNVT0 with major:minor devnode 199:6, and on port-open connects to 192.168.1.10:5000 where it expects a RFC2217-compliant server)
http://lpccomp.bc.ca/remserial/
Written in C. Available for many distros. Can create local devicefile-like interface, linked to a pseudoterminal (pty).
No support for ioctl interception.
Written in C. Available for many distros. Can create local devicefile-like interface, linked to a pseudoterminal (pty).
Does not seem to be able to intercept ioctls.
TODO: check
http://www.dest-unreach.org/socat/contrib/socat-rfc2217.html
Not a device driver, only a userspace terminal for handling local serial ports and some connections to remote devices. A versatile tool for simplest terminal operations, supports hex.
todo: look at its proprietary extensions for GPIOs, implement the calls to ttyvnt
https://www.hw-group.com/software/hw-vsp3-virtual-serial-port
https://freevirtualserialports.com/
The original implementation of serial port did not support arbitrary baudrates, uses the termios structure. Newer variant is not limited by the hardcoded baudrate list, uses extended termios2 structure and modified ioctl calls.
ttynvt of course did not support these. (Because nothing can be perfect. Oh well, let's start with good-enough.)
To obtain support for arbitrary baud rates, and for compatibility with newer software and it's ioctl calls, the missing ioctls were crudely hacked into the original ttynvt implementation.
Experiences were obtained with handling ioctl calls, allowing to further extend the virtual port functionality - add "vendor-specific" command sequences, eg. for handling various GPIO and bitbanging modes on USB-serial dongles.
Server-side then also has to be modified for support of such.
Virtual port servers can be implemented with relative ease, to appear like a RFC2217-compliant remote serial port (and to appear like a real local serial port in the system). Emulation of various devices (eg. position-faking GPS, for testing or "testing" purposes) is one of the options. Or maybe a protocol translator between one vendor's software and another vendor's device. (This can also be done locally, using the same CUSE trick ttynvt does to act as a proxy between a local serial port, but a python implementation of just the server side may be easier to both write and debug.)
The extensions for control signals can be used for including such out of band data over RS485. A two-port microcontroller to translate messages, split data stream from command stream, handle packetization/addressing for the bus.
esptool.py was crashing on port open:
in slip_reader(): replace
read_bytes = port.read(1 if waiting == 0 else waiting) try: read_bytes = port.read(1 if waiting == 0 else waiting) except (OSError, serial.serialutil.SerialException): print("[caught error: no data]") continue - original files
- modified files
- patch file with changes