I translated scsi_mid_low_api.txt for my study, so make a note. Introduction This document outlines the interface between Linux SCSI mid level and SCSI lower level drivers. Lower level drivers (hereinafter LLDs) are called by various things such as host bus adapter (hereinafter HBA) driver and host driver (hereinafter HD). In this context, "host" means a bridge that connects an IO bus (eg PCI or ISA) to a single SCSI Initiator Port that resides on a SCSI transport. The "initiator" port (SCSI terminology, see SAM-3) sends SCSI commands to the "target" SCSI port (eg disk). There may be many LLDs on a running system, but only one per hardware type applies. Most LLDs can control one or more SCSI HBAs. Some HBAs contain multiple hosts.
SCSI transport can also be an external bus that already has its own subsystem in Linux. (Example: USB and ieee1394). In such cases, the LLD in the SCSI subsystem becomes a software bridge to the subsystems of other drivers. Examples are the usb-storage driver (located in dirvers / usb / storage) and the iee1394 / sbp2 driver (located in drivers / ieee1394).
For example, the aic7xxx LLD controls the Adaptec SCSI parallel interface (SPI) controller that employs the company's 7xxx chip series. Aic7xxx LLD can be built into the kernel or loaded as a module. Only one aic7xxx LLD runs on a Linux system, but it may control many HBAs. These HBAs may exist as PCI daughterboards or be built into the motherboard (or both). Some aic7xxx-based HBAs are dual controllers. Therefore, it represents two hosts. Like the latest HBAs, each host in aic7xxx has its own PCI device address. [One-to-one correspondence between SCSI hosts and PCI devices is common, but not required. (Example: When using ISA adpter)]
The SCSI mid layer isolates the LLD from other layers, such as the SCSI upper layer driver and the block layer driver.
This document is roughly consistent with Linux kernel version 2.6.8.
Documentation The SCSI documentation directory is in the Linxu source tree, usually Documentation / scsi. Most documents are plain text (ie ASCII). This file is named scsi_mid_low_api.txt and can be found in that directory. A more recent copy of this document http://web.archive.org/web/20070107183357rn_1/sg.torque.net/scsi/ You may find it at. Many LLDs are documented there. (Example: Aic7xxx.txt). The SCSI mid level is briefly mentioned in scsi.txt, which contains a url to a document that describes the Linux Kernel 2.4 family of SCSI subsystems. The two upper level drivers have documents in that directory. That is, st.txt (SCSI tape driver) and scsi-generic.txt (for sg driver).
Some LLDs documentation or urls may be found in the C source code or in the same directory as the C source code. For example, to find the URL for the USB mass storage driver, look in the / usr / src / linux / drivers / usb / storage directory.
Driver structure Traditionally, LLDs for SCSI subsystems consist of at least two files in the drivers / scsi directory. For example, a driver called xyz has a header file xyz.h and a source file xyz.c. [Actually, there is no clear reason why you can't write everything in one file. The header file may be extra]. Some drivers that are ported to different operating systems have more than one file. For example, the aic7xxx driver has separate files for generic and OS-specific (eg FreeBSD and Linux). Such drivers tend to create their own directories under the drivers / scsi directory.
The following files (found in the drivers / scsi directory) should be noted when adding a new LLD to Linux. Makefile and Kconfig. It's probably best to find out how your existing LLD is configured.
Changes have been introduced into this interface as the 2.5 series development kernel evolved into the 2.6 series production series. One example of this is the driver initialization code that enables the two models. The old initialization is based on the host found when loading the HBA driver. Similar ones can be found in the 2.4 series. This is called the "passive" initialization model. The new model allows the HBA to be hotplugged and unplugged during the lifetime of the LLD. And this is called the "hotplug" initialization model. The new model is preferred because it treats both traditional SCSI devices that are permanently connected and modern SCSI devices that are hot-plugged (eg digital cameras with USB or IEEE1394 connections) in the same way. Both initialization models are discussed in the sections below.
There are various ways to interface the LLD to the SCSI subsystemm: a) Execute the function directly from the Mid level b) Via the function pointer for function registration provided by Mid level. Mid level will execute these functions at some point in the future. LLD provides implementations for these functions. c) Direct access to instances of well-known data structures maintained by Mid level
a) The functions in the group are listed in the "Mid level supplied functions" chapter below.
b) The group functions are listed in the "Interface functions" chapter below. Those function pointers exist in the "struct scsi_host_template" and are passed by scsi_host_alloc () (* Note 1). Function interfaces that LLD does not want to provide set the corresponding members of scsi_host_template to NULL. The definition of an instance of scsi_host_template in file scope sets a function pointer that has not been explicitly initialized to NULL.
c) Be careful when using groups, especially in a "hotplug" environment. The LLD must be aware of the lifetime of the instance shared with the mid level and other layers.
All functions defined in the LLD and all data defined in the file scope should be static. For example, the slave_alloc () function defined in the LLD named xxx can be defined as "static int xxx_slave_alloc (struct scsi_device * sdev) {/ * code * /}".
(* Note 1) The scsi_host_alloc () function is almost always replaced by the scsi_register () function. The scsi_register () and scsi_unregister () functions remain to maintain support for the passive initialization model.
Hotplug initialization model This model controls when the LLD introduces and removes SCSI hosts from the SCSI subsystem. The Host is deployed as fast as the driver initialization and is removed as late as the driver is shut down. The driver typically responds to a sysfs probe () callback indicating that an HBA has been detected. After verifying that the new device is what the LLD wants to control, the LLD initializes the HBA and then registers the new host at the SCSI mid level.
During LLD initialization, the driver should register itself on the appropriate IO bus where the HBA is expected to be discovered. (Example: PCI bus). This is probably possible via sysfs. Some driver parameters, especially those that are writable after the driver is loaded, can also be registered with sysfs at this point. The SCSI mid level will recognize the LLD when it registers the first HBA.
After some time, the LLD recognizes the HBA and the subsequent general call sequence between the LLD and the mid level. This example shows the mid level that scans for newly introduced HBAs where only the first two of the three SCSI devices return a response.
HBA PROBE: assume 2 SCSI devices found in scan
LLD mid level LLD
===-------------------=========--------------------===------
scsi_host_alloc() -->
scsi_add_host() ---->
scsi_scan_host() -------+
|
slave_alloc()
slave_configure() --> scsi_adjust_queue_depth()
|
slave_alloc()
slave_configure()
|
slave_alloc() ***
slave_destroy() ***
------------------------------------------------------------
If LLD wants to adjust the default queue settings, it invokes scsi_adjust_queue_depth () in the slave_configure () routine.
*** For scsi devices where mid level does not respond after trying a scan, the slave_alloc (), slave_destroy () pair is called.
When the HBA is removed, it may be part of the normal shutdown process associated with the unloaded LLD module, or it may be a response to the "hot unplug" intended by the sysfs remove () callback being invoked. Maybe. In either case, the sequence is the same.
HBA REMOVE: assume 2 SCSI devices attached
LLD mid level LLD
===----------------------=========-----------------===------
scsi_remove_host() ---------+
|
slave_destroy()
slave_destroy()
scsi_host_put()
------------------------------------------------------------
It may be useful for LLD to follow an instance of the Scsi_Host structure (a pointer to the return value of scsi_host_alloc ()). Such instances are owned by mid-level. Instances of the Scsi_Host structure are released via scsi_host_put () when the reference count reaches zero.
Hot unplugging the HBA that controls the disk processing the SCSI commands on the mounted filesystem is an interesting situation. Reference counting logic has been introduced in mid-level to address many issues related to mid-level. See the reference count chapter below.
The concept of hotplug may be extended to SCSI devices. Currently, when the HBA is added, the scsi_scan_host () function scans the SCSI device attached to the HBA's SCSI transport. With newer SCSI transports, the HBA may notice new SCSI devices after the scan process is complete. The LLD can use the following sequence to make the mid-level aware of SCSI devices.
SCSI DEVICE hotplug
LLD mid level LLD
===-------------------=========--------------------===------
scsi_add_device() ------+
|
slave_alloc()
slave_configure() [--> scsi_adjust_queue_depth()]
------------------------------------------------------------
In a similar way, the LLD may become aware that the SCSI device has been removed or the connection to the SCSI device has been interrupted. Some existing SCSI transports (eg SPI) may not be aware that the SCSI device has been removed until the SCSI command to offline the SCSI device issued by mid-level fails. LLDs that detect SCSI device removal can begin removal from the upper layer using the following sequence:
SCSI DEVICE hot unplug
LLD mid level LLD
===----------------------=========-----------------===------
scsi_remove_device() -------+
|
slave_destroy()
------------------------------------------------------------
It may be useful for LLD to follow an instance of the scsi_device structure (pointers are passed as parameters to the slave_alloc () and slave_configure () callbacks). Such instances are owned by mid-level. The scsi_device structure is released after slave_destroy ().
Passive initialization model These older LLDs contain a file called "scsi_module.c" in their source code. For that file to work, you need to define an instance of the scsi_host_template structure called driver_template. Here's a typical code sequence usage for this model: static struct scsi_host_template driver_template = { ... }; #include "scsi_module.c" The Scsi_module.c file contains two functions: --Init_this_scsi_driver () Executed when LLD is initialized (i.e at startup or module load) --Exit_this_scsi_driver () Executed when LLD is shut down (when i.e module is unloaded) Note: These functions are tagged with the __init and __exit modifiers, so LLD should not be called explicitly. (Because the kernel executes it)
This is an example where two hosts are recognized (detect () returns 2) and a SCSI bus scan is performed on each host to discover one SCSI device. (The second SCSI device didn't respond).
LLD mid level LLD
===----------------------=========-----------------===------
init_this_scsi_driver() ----+
|
detect() -----------------+
| |
| scsi_register()
| scsi_register()
|
slave_alloc()
slave_configure() --> scsi_adjust_queue_depth()
slave_alloc() ***
slave_destroy() ***
|
slave_alloc()
slave_configure()
slave_alloc() ***
slave_destroy() ***
------------------------------------------------------------
Mid-level turns off tagged queuing, calls scsi_adjust_queue_depth () and sets the queue length to "cmd_per_lun" for that host. These settings can be overridden with slave_configure () provided by LLD. *** A pair of slave_alloc () and slave_detroy () is called for SCSI devices whose mid level tried to scan and did not respond.
Below is the sequence in which the LLD shuts down.
LLD mid level LLD
===----------------------=========-----------------===------
exit_this_scsi_driver() ----+
|
slave_destroy()
release() --> scsi_unregister()
|
slave_destroy()
release() --> scsi_unregister()
------------------------------------------------------------
LLD does not need to define slave_destroy () (i.e. it is optional).
The disadvantage of the "passive initialization" model is that registering and unregistering hosts is tied to LLD initialization and shutdown. Once the LLD is initialized, it is not easy to add newly emerging hosts (via e.g hotplug) without shutting down or reinitializing the driver. It may be possible to write an LLD for both initialization models.
Reference Counting A reference count infrastructure has been added to the Scsi_Host structure. This effectively extends ownership of the Scsi_Host instance through the various SCSI layers that utilize them. Previously, such instances were exclusively owned by mid level. LLDs usually do not need to manipulate these reference counts directly, but in some cases they do.
There are three interesting reference count functions related to the Scsi_Host structure. --scsi_host_alloc (): Returns a pointer to a new instance of the Scsi_Host structure with the reference count set to 1. ^^ --scsi_host_get (): Adds 1 to the reference count for a given instance. --scsi_host_put (): Subtracts the reference count for a given instance by 1. If the reference count reaches 0, release the given instance
A reference count infrastructure has been added to the Scsi_device structure. Ownership of Scsi_device instances is effectively extended through the various SCSI layers that utilize them. Previously, such instances were exclusively owned by mid level. Look at the access functions declared towards the end of Include / scsi / scsi_device.h. If the LLD wants to keep a copy of the pointer to the Scsi_device instance, it should use scsi_device_get () to push up its reference count. When the pointer is finished, you can use scsi_device_put () to reduce its reference count (and potentially remove it).
^^ The Scsi_Host structure actually has two reference counts that are manipulated in parallel by these functions. (Shost_dev and host_gendev?)
Conventions First, ideas about Linus Torvalds C's coding style can be found in the Documentation / CodingStyle file.
Next, there is a move to ban typedefs that introduce synonyms for struct tags. Both can still be found in the SCSI subsystem, but such typedefs have been moved to a single file scsi_typedefs.h to make future removal easier. For example: "typedef struct scsi_cmnd Scsi_Cmnd;"
Also, most C99 enhancements encourage extensions for which they are supported by the associated gcc compiler. Therefore, C99-style struct and array initializers are encouraged when needed. Don't overdo it, VLAs is not yet properly supported. The exception to this is to use "//" style comments. / * * / Style comments are still preferred on Linux.
Well-written, tested, and documented code does not need to be reformulated to follow the conventions above. For example, the aic7xxx driver comes to Linux from FreeBSD and Adaptec laboratories. I'm sure FreeBSD and Adaptec have their own coding conventions.
Mid level supplied functions These functions are provided by the SCSI mid level for use by LLD. The names of these functions (ie the entry points) are exported so that the LLD can access them. The kernel loads and prepares the SCSI mid level for initialization before any LLD is initialized. The following functions are arranged alphabetically and all their function names start with "scsi_".
Summary:
scsi_activate_tcq - turn on tag command queueing
scsi_add_device - creates new scsi device (lu) instance
scsi_add_host - perform sysfs registration and set up transport class
scsi_adjust_queue_depth - change the queue depth on a SCSI device
scsi_bios_ptable - return copy of block device's partition table
scsi_block_requests - prevent further commands being queued to given host
scsi_deactivate_tcq - turn off tag command queueing
scsi_host_alloc - return a new scsi_host instance whose refcount==1
scsi_host_get - increments Scsi_Host instance's refcount
scsi_host_put - decrements Scsi_Host instance's refcount (free if 0)
scsi_partsize - parse partition table into cylinders, heads + sectors
scsi_register - create and register a scsi host adapter instance.
scsi_remove_device - detach and remove a SCSI device
scsi_remove_host - detach and remove all SCSI devices owned by host
scsi_report_bus_reset - report scsi _bus_ reset observed
scsi_scan_host - scan SCSI bus
scsi_track_queue_full - track successive QUEUE_FULL events
scsi_unblock_requests - allow further commands to be queued to given host
scsi_unregister - [calls scsi_host_put()]
Details:
abridgement
Interface Functions Interface functions are defined by LLD and their function pointers are placed on instances of the scsi_host_template structure assigned by scsi_host_alloc () [or scsi_register () / init_this_scsi_driver ()]. Some are mandatory. Interface functions should be declared static. The accepted convention is that the driver "xyz" declares its slave_configure () function as static int xyz_slave_configure (struct scsi_device * sdev). And so are all the interface functions listed below.
A pointer to this function should be placed in the slave_configure member of the scsi_host_template structure instance. Pointers to such instances should be passed to mid level scsi_host_alloc () [or scsi_register () / init_this_scsi_driver ()].
Interface functions are also written just above the definition of the scsi_host_template structure in the include / scsi / scsi_host.h file. In some cases, details can be found below that in scsi_host.h.
The interface functions are listed below in alphabetical order.
Summary:
bios_param - fetch head, sector, cylinder info for a disk
detect - detects HBAs this driver wants to control
eh_timed_out - notify the host that a command timer expired
eh_abort_handler - abort given command
eh_bus_reset_handler - issue SCSI bus reset
eh_device_reset_handler - issue SCSI device reset
eh_host_reset_handler - reset host (host bus adapter)
info - supply information about given host
ioctl - driver can respond to ioctls
proc_info - supports /proc/scsi/{driver_name}/{host_no}
queuecommand - queue scsi command, invoke 'done' on completion
release - release all resources associated with given host
slave_alloc - prior to any commands being sent to a new device
slave_configure - driver fine tuning for given device after attach
slave_destroy - given device is about to be shut down
Details:
abridgement
Data Structures struct scsi_host_template There is one "struct scsi_host_template" per LLD **. It is initialized as a static file scope in the driver header file. Members that are not explicitly initialized that way are set to 0 or NULL. The members of interest are: name --Driver name (spaces allowed, but up to 80 characters) proc_name-used by sysfs in one of the <proc_name> and "drivers" directories in "/ proc / scsi / <proc_name> / <host_no>". In addition, "proc_name" should contain only characters that are accepted by Unix filenames. ( queuecommand) () --mid level is the main callback used to inject SCSI commands into the LLD. The scsi_host_template structure is defined and commented on in include / scsi / scsi_host.h.
*** In extreme situations, a driver may have multiple instances if one driver controls several different classes of hardware. (e.g. LLD handles both ISA and PCI cards and has a separate scsi_host_template structure for each class).
struct Scsi_Host There is one Scsi_Host structure for each LLD-controlled host (HBA). The Scsi_Host structure has many members in common with the "struct scsi_host_template". When an instance of a new Scsi_Host structure is created (created by scsi_host_alloc () in hosts.c), their common members are initialized from an instance of the driver's scsi_host_template structure. The members of interest are:
host_no-The only system-wide number to identify this host. Values are assigned in ascending order from 0. can_queue --must be greater than 0. Do not send more commands to the adapter than the value of can_queue. this_id --Represents the ID of the SCSI host (initiator). Set to -1 if unknown. sg_tablesize --Maximum number of scatter gather elements allowed by the host. If 0, it means that the host does not support scatter gather. max_sectors --The maximum number of sectors allowed by a single SCSI command (usually 512 bytes). The initial value of 0 leads to the SCSI_DEFAULT_MAX_SECTORS (defined in scsi_host.h) setting, which currently sets 1024. So far, the maximum transfer length of a disk is 512KB when max_sectors is not defined. Please note that this size may not be sufficient for uploading disk firmware. cmd_per_lun --Maximum number of commands that can be queued to a host-controlled device. Overwrite by LLD calling scsi_adjust_queue_depth (). When unchecked_isa_dma -1 =>, only the lower 16MB of RAM is used (ISA DMA addressing limit), and when 0> =, 32bit or more DMA address space can be fully used. use_clustering --1 => mid level SCSI commands are mergeable, 0 => SCSI command merging is not allowed hostt --A pointer to the driver's scsi_host_template structure from an instance of the generated Scsi_Host structure. The name of hostt-> proc_name -LLD. This is the name of the driver used by sysfs. transportt --a pointer to the scsi_transport_template instance (if any). FC and SPI transports are currently supported. sh_list-A two-way linked list to all Scsi_Host instances. (Currently and in ascending order of host_no). my_devices --A bidirectional linked list to scsi_device structures that belong to this host. hostdata [0] --Reserved area for LLD at the end of the Scsi_Host structure. The size is set by the second argument of scsi_host_alloc () or scsi_register (). vendor_id --A unique value that identifies the vendor-provided LLD for Scsi_Host. Mostly used to validate vendor hot water message requests. The value consists of the identifier type and the vendor-specific value. See scsi_netlink.h for a description of valid formats.
The scsi_host structure is defined in include / scsi / scsi_host.h.
struct scsi_device Generally, there is one instance of this structure for each SCSI LU on a host. SCSI devices connected to a host are uniquely identified by their channel number, target ID, and logical unit number (lun). This structure is defined in include / scsi / scsi_device.h.
struct scsi_cmnd An instance of this structure propagates SCSI commands to the LLD and returns a response to the mid level. SCSI mid level does not queue SCSI commands to LLD beyond the value intended by scsi_adjust_queue_depth () (or cmd_per_lun in the Scsi_Host structure). At least one instance of the scsi_cmnd structure will be available for each SCSI device. The members of interest are:
cmnd --Array holding SCSI commands cmnd_len --SCSI command array length (in bytes) sc_data_direction --The direction of data transfer during the data phase. See "enum dma_data_direction" in include / linux / dma-mapping.h request_bufflen --Number of data bytes to transfer (0 means no data to transfer) use_sg-== 0-> Do not use the scatter list. That is, the data transferred from / to the request buffer-> 0-> scatter gather list of request_bufffer using the use_sg element (actually an array) request_buffer --The data buffer or scatter gather list that depends on the setting of use_sg which one. Scatter gather elements are defined by the struct scatterlist in include / asm / scatterlist.h. done --A function pointer that should be invoked by the LLD when the SCSI command completes (successfully or otherwise). If the LLD accepted the command (i.e queuecommand () returned or returned 0), it should only be called by the LLD. LLD may start done before queuecommand () finishes. LLD should be set by before calling result --done. A value of 0 means that the command was successful. (And all the data was transferred from / to the SCSI target device.) Result is a 32-bit unsigned integer viewed as 4 related bytes. The SCSI status value is LSB. See the constants associated with the status_byte (), msg_byte (), host_byte (), driver_byte () macros in Include / scsi / scsi.h. sense_buffer --Array to be written when SCSI status (LSB of result) is set to CHECK CONDITION (2) (maximum length: SCSI_SENSE_BUFFERSIZE bytes). When CHECK CONDITION is set, if the upper 4 bits of sense_buffer [0] have a value of 7, mid level assumes that the sense_buffer array contains a valid SCSI sense buffer. Otherwise, mid level will issue a REQUESET SENSE SCSI command to get a sense buffer. The latter strategy is prone to error in the presence of command queuing. So LLD should always be auto-sense. device --A pointer to the scsi_device object associated with this command resid --LLD should be set to this signed integer minus the requested transfer length (i.e request_buffer) minus the number of bytes actually transferred. The resid is preset to 0, and if underrun (overrun is rare) cannot be detected, LLD can ignore it. If possible, LLD should set resid before launching done. The most interesting case is data transfer from underrun SCSI target devices (e.g READs). underflow --LLD should put (DID_ERROR_ << 16) in result if the number of bytes actually transferred is less than this value. Not many LLDs implement this check and spit out error messages to the log rather than reporting DID_ERROR. Better for LLD is to implement resid.
LLD recommends setting resid for data transfer from SCSI target devices (eg READs). Setting the resid is especially important for data transfers that have a sense key of MEDIUM_ERROR and HARDWARE ERROR (RECOVERD ERROR if possible). In these cases you don't know how much the LLD will accept the data and then the safest approach is to not accept the data. For example: LLD may use these helper functions to indicate that no valid data has been received:
scsi_set_resid(SCpnt, scsi_bufflen(SCpnt));
SCpnt is a pointer to the scsi_cmnd object. The resid is set this way to indicate that only three 512-byte blocks have been received.
scsi_set_resid(SCpnt, scsi_bufflen(SCpnt) - (3 * 512));
The scsi_cmnd structure is defined in include / scsi / scsi_cmnd.h.
Locks An instance of the Scsi_Host structure has a spin_lock called Scsi_Host :: default_lock that is initialized with scsi_host_alloc (). Within the same function, the Scsi_Host :: host_lock pointer is initialized to point to default_lock. The lock and unlock operations are then performed by the mid level using the Scsi_Host :: host_lock pointer. The previous driver was able to override host_lock, but this is no longer allowed.
Autosense Autosense (or auto-sense) is defined in the SAM-2 document that when the CHECK_CONDITION state occurs, the automatic response of sense data to the application client coincides with the completion of the SCSI command. LLDs should play autosense. This should be done when the LLD detects a CHECK CONDITION by each: a) Instruct the SCSI protocol to play more data in such a response. b) Alternatively, the LLD issues the REQUEST SENSE command to itself.
Either way, when CHECK CONDITION is recognized, mid level checks struct scsi_cmnd :: sense_buffer [0] to determine if LLD plays autosense. If this byte has the upper 4 bits of 7 (meaning 0b0111? Or 0xf), then it is assumed that autosense has occurred. If it has a different value (and this byte is initialized to 0 before each command), mid level issues the REQUESET SENSE command.
If a queued command exists, nexus, which keeps the sense buffer data from the command that fails until the next REQUESET SENSE, may be out of sync. This is the best reason LLD plays autosense.
Recommended Posts