userdma(2) — CX/UX

NAME

userdma − prepare a buffer for DMA transfers

SYNOPSIS

#include <sys/types.h>
#include <sys/mman.h>

int userdma (addr, len, cmd, flags, vec, nvec)
caddr_t addr;
size_t len;
int cmd;
int flags;
struct dmavec ∗vec;
int nvec;

DESCRIPTION

Some real-time applications need a more direct, lower-latency method of accessing I/O devices than can be provided with the read(2), write(2), and ioctl(2) system calls. By mapping portions of their address spaces onto an I/O device’s register set (see shmbind(2)), these applications can remove all system call overhead from device access. However, making use of an I/O device’s DMA capabilities directly from user mode introduces further problems. Standard DMA hardware operates at the physical memory level, bypassing memory management units and sometimes even data caches. The following are required for DMA transfers to or from the virtual address space of an application:

1) The application’s buffer must be locked into physical memory; that is, the buffer must be resident and the virtual to physical mappings must not be allowed to change.

2) The application needs to know the physical location of the buffer.

3) CPU access and I/O access to the buffer must be coherent.

4) The virtual pages containing the buffer need to be marked "used" and for DMA reads, "modified."

userdma() performs the services mentioned above. Addr is the address of the first byte of an I/O buffer, and len is the number of bytes in the buffer. Cmd and flags specify the particular operation to be performed on the buffer. Vec points to an array of dmavec structures that are used to return the physical of the buffer, and nvec is the number of elements in the array.

Cmd must contain one of the following values:

USERDMA_LOCK Faults the pages in the range [addr, addr+len) into physical memory, brings physical memory and CPU data caches up to date, and increments the number of locks on each page. A page’s virtual to physical mapping will not change (see the WARNINGS below) as long as the number of locks on it is nonzero. The caller must either have an effective user ID equal to that of the super-user or must have the ACC_PLOCK privilege to use this command.

The following flags may be specified:

USERDMA_WRITE Indicates that the buffer will be used to write to a device. The pages containing the buffer will be checked for read access and marked as "used."

USERDMA_READ Indicates that the buffer will be used to read from a device. The pages containing the buffer will be checked for write access and marked as "used" and "modified."

These flags are used to maintain cache coherence in an architecture-independent fashion. Suppose, for example, that an application re-uses a single buffer in several DMA writes. Before each write, the application fills the buffer with new data. On architectures possessing write-back data caches that do not snoop DMA traffic, the cache modes of the pages containing the buffer must be altered to keep memory and cache coherent. No changes are needed on architectures that snoop DMA traffic or that use write-through data caches. Because this command can alter the cache modes of a page, applications are advised to separate high-use data and DMA buffers into different pages for maximum performance.

If neither flag is set, the buffer is locked into physical memory and the cache modes of the pages containing the buffer are NOT modified.

Nvec is the number of elements in the array of dmavec structures referenced by vec. If nvec is not zero, userdma() returns the physical location of the buffer in the array of vectors. Each vector describes a maximally contiguous physical buffer fragment. The functional value of userdma() is the number of vectors actually returned, which can be as few as one if the entire buffer is physically contiguous, and as many as one per page otherwise. A dmavec structure has the following fields:

unsigned int dma_paddr; /∗ Physical address and ... ∗/
unsigned int dma_plen; /∗ length of buffer fragment. ∗/

If nvec is zero, userdma() does not return the physical location of the buffer.

USERDMA_UNLOCK Decrements the number of locks on each page in the range [addr, addr+len). When a page’s lock count reaches zero, its cache mode reverts to its original state, and its virtual to physical mapping can change. The caller must either have an effective user ID equal to that of the super-user or must have the ACC_PLOCK privilege to use this command.

The flags used to lock the buffer should also be used to unlock the buffer. vec and nvec must be zero.

USERDMA_VTOP Returns the physical location of the buffer [addr, addr+len) in a manner similar to USERDMA_LOCK. The buffer must be locked. This command only performs virtual to physical address translation; it does NOT lock the buffer into memory.

flags must be zero.

EXAMPLES

To compute the number of pages in the range [addr, addr+len), use the formula

nvec = ((unsigned)(addr+len-1) / nbpp)
- ((unsigned)(addr) / nbpp) + 1;

where nbpp is the number of bytes in a page (see getpagesize(2)). The following call to userdma() locks the buffer and returns its physical location in vec -- an array of dmavec structures that has at least nvec elements.

nvec = userdma (addr, len, USERDMA_LOCK, 0, vec, nvec);

Upon return, nvec contains the number of dmavec structures actually used. When the buffer is no longer needed, the following call unlocks it.

userdma (addr, len, USERDMA_UNLOCK, 0, 0, 0);

WARNINGS

After a fork(2), the child process does not inherit its parent’s page locks.

Fork(2) generally uses copy-on-write techniques to reduce the amount of data that must be copied from the parent process to the child process. Real copies, however, are made of locked pages in order to avoid copy-on-write protection faults in the parent. For maximum performance, applications are advised to avoid fork(2) when large portions of their address spaces are locked.

The physical location of a buffer mapped to a local memory pool will change if the process migrates to a CPU not attached to that pool (see memadvise(2) and mpadvise(2)). This form of migration only occurs in mpadvise(2) at the application’s request; the operating system’s load balancer never migrates a process out of a local memory pool.

The number of locks that can be simultaneously applied to a single page is limited to an implementation-dependent value.

RETURN VALUE

Upon successful completion of USERDMA_LOCK or USERDMA_VTOP, the value returned by userdma() is the number of dmavec structures used to describe the physical location of the buffer. Upon successful completion of USERDMA_UNLOCK, 0 is returned. Otherwise, a value of −1 is returned and errno is set to indicate the error.

ERRORS

userdma() will fail if any of the following are true:

[EPERM] The calling process does not have the appropriate privilege to perform the requested operation.

[EINVAL] Cmd or flags was not one of the values documented above.

[EINVAL] There are an insufficient number of dmavec structures to describe the physical location of the buffer.

[EFAULT] Some of the addresses in the range [addr, addr+len) are invalid.

[EFAULT] The USERDMA_READ flag was specified on the USERDMA_LOCK command and the caller does not have write access to some of the addresses in the range [addr, addr+len).

[EFAULT] The USERDMA_VTOP command was specified and some of the addresses in the range [addr, addr+len) are not locked.

[EFAULT] Vec contains an invalid address.

[EAGAIN] There are insufficient resources.

[EBUSY] The calling process has attempted to place more locks on a page than is supported by the implementation.

Museum

Related Articles