請教e1000中DMA傳輸?shù)膯栴}

tqkangkang

想通過修改e1000驅(qū)動，實現(xiàn)把網(wǎng)卡接收到的包以DMA方式傳到指定的buffer里面。然而卻找不到e1000是如何與dma交互的，請各位大蝦指教，謝謝！

prc

直接memcpy就好了，為什么還要dma

bekars

pci_map_single函數(shù)建立的DMA映射，pci_unmap_single函數(shù)進(jìn)行DMA傳送

2樓的回去看看驅(qū)動再說話

albcamus

原帖由 bekars 于 2007-4-25 10:03 發(fā)表于 3樓  
pci_map_single函數(shù)建立的DMA映射，pci_unmap_single函數(shù)進(jìn)行DMA傳送

2樓的回去看看驅(qū)動再說話

之前是不是要pci_set_master?

prc

pci_unmap_single進(jìn)行dma傳輸？別逗了....

看看pci_map_single和pci_unmap_single分別是怎么實現(xiàn)的：
---------------------------------------------------------------------------
linux/include/asm-generic/pci-dma-compact.h
static inline dma_addr_t
pci_map_single(struct pci_dev *hwdev, void *ptr, size_t size, int direction)
{
    return dma_map_single(hwdev == NULL ? NULL : &hwdev->dev, ptr, size, (enum dma_data_direction)direction);
}

/linux/include/asm-i386/dma-mapping.h
------------------------------------------------------------
static inline dma_addr_t
dma_map_single(struct device *dev, void *ptr, size_t size,
           enum dma_data_direction direction)
{
    BUG_ON(direction == DMA_NONE);
    flush_write_buffers();
    return virt_to_phys(ptr);
}

static inline dma_addr_t
dma_map_single(struct device *dev, void *ptr, size_t size,
           enum dma_data_direction direction)
{
    BUG_ON(direction == DMA_NONE);
    flush_write_buffers();
    return virt_to_phys(ptr);
}

static inline void
dma_unmap_single(struct device *dev, dma_addr_t dma_addr, size_t size,
         enum dma_data_direction direction)
{
    BUG_ON(direction == DMA_NONE);
}

dma_map_single返回ptr的物理地址，dma_unmap_single什么都不做。
因為外設(shè)在訪問內(nèi)存的時候需要的是物理地址，所以driver才需要做用函數(shù)pci_map_single將虛地址轉(zhuǎn)成物理地址。這段地址是由CPU和外設(shè)共享的。一般情況下CPU只做讀操作，寫入是由外設(shè)完成的。

現(xiàn)在的網(wǎng)卡大多數(shù)都是采用主動DMA的方式。也就是當(dāng)網(wǎng)卡從網(wǎng)線上接收到數(shù)據(jù)之后，就會自己啟動dma將數(shù)據(jù)從網(wǎng)卡內(nèi)部的FIFO傳送到配置寄存器指定的內(nèi)存地址。當(dāng)一個數(shù)據(jù)包接收完成之后，產(chǎn)生一個中斷通知driver進(jìn)行處理。整個過程不需要cpu進(jìn)行干涉。當(dāng)driver接收到中斷的時候，數(shù)據(jù)已經(jīng)在內(nèi)存里存放好了。

對linux系統(tǒng)來說，大多數(shù)網(wǎng)卡driver為了提高處理效率，都會將skb->data指向這塊共享內(nèi)存中，這樣可以減少一次內(nèi)存拷貝操作。
在e1000_clean_rx_irq函數(shù)里在調(diào)用 netif_rx 之前直接訪問skb->data就可以了...這就是收到的數(shù)據(jù)包。

如果真的要自己指定地址，修改 alloc_rx_buf 里面的實現(xiàn)代碼就好了

bekars

就會自己啟動dma將數(shù)據(jù)從網(wǎng)卡內(nèi)部的FIFO傳送到配置寄存器指定的內(nèi)存地址


請問這個配置寄存器是誰設(shè)置的，什么時候設(shè)置的？


我看的是ARM的內(nèi)核代碼，和i386的有些區(qū)別，不同平臺的DMA實現(xiàn)應(yīng)該很不同，但是你開始說的直接memcpy確實沒有根據(jù)，現(xiàn)在的網(wǎng)卡驅(qū)動都是用DMA來傳輸數(shù)據(jù)的。驅(qū)動收包之后就不會再做內(nèi)存拷貝的動作了，skb保存在內(nèi)存的環(huán)形隊列中，供協(xié)議棧處理。

[ 本帖最后由 bekars 于 2007-4-25 15:31 編輯 ]

bekars

ixp425的pci_map函數(shù)

dma_addr_t ixp425_pci_map_single(void *virt, size_t size, int direction)
{
        dma_addr_t dma_addr;
        unsigned long flags;
        void *safe;

        DBG("ixp425_map_single(virt=%p,size=%d,dir=%x)\n",
                virt, size, direction);

        dma_addr = virt_to_bus(virt);

        if(((u32)virt + size) >= (CONFIG_KERNEL_START + SZ_64M)) {
                safe = alloc_safe_buffer(virt, size, &dma_addr);
                if (!safe) {
                        printk("%s: Could not allocate safe buffer",
                                        __FILE__);
                        return 0;
                }

                DBG("unsafe buffer %p (phy=%p) mapped to %p (phy=%p)\n", virt,
                        (void *)virt_to_phys(virt), safe, (void *)dma_addr);

                /*
                 * Only need to copy if DMAing to device
                 */
                if((direction == PCI_DMA_TODEVICE) ||
                   (direction == PCI_DMA_BIDIRECTIONAL)) {
                        memcpy(safe, virt, size);
                }
                consistent_sync(safe, size, direction);
        }
        else
                consistent_sync(virt, size, direction);

        return dma_addr;
}

void ixp425_pci_unmap_single(dma_addr_t dma_addr, size_t size, int direction)
{
        void *safe, *unsafe;
        unsigned long flags;
        struct safe_buffer *safe_buf;

        DBG("ixp425_unmap_single(ptr=%p, size=%d, dir=%x)\n",  
                (void *)dma_addr, size, direction);

        if ((safe_buf = find_safe_buffer(dma_addr, &unsafe))) {
                if((direction == PCI_DMA_FROMDEVICE) ||
                   (direction == PCI_DMA_BIDIRECTIONAL)) {
                        DBG("copyback unsafe %p, safe %p, size %d\n", unsafe, safe_buf->safe, size);
                        consistent_sync(safe_buf->safe, size, direction);
                        memcpy(unsafe, safe_buf->safe, size);
                }
       
                free_safe_buffer(safe_buf);
        } else {
                /*
                 * Assume this is normal memory.  We have a possible
                 * OOPs here if someone sends us a bad dma_addr_t.
                 */
                unsafe = bus_to_virt(dma_addr);
                consistent_sync(unsafe, size, direction);
        }
}


這個地方也想請教一下，一直對DMA硬件如何實現(xiàn)的不清楚：
1）i386平臺的DMA的寄存器是如何設(shè)置的？
2）網(wǎng)卡中的數(shù)據(jù)是如何通過DMA傳到內(nèi)存中的？

[ 本帖最后由 bekars 于 2007-4-25 16:10 編輯 ]

bekars

Part Id - Streaming DMA mappings
--------------------------------

dma_addr_t
dma_map_single(struct device *dev, void *cpu_addr, size_t size,
                      enum dma_data_direction direction)
dma_addr_t
pci_map_single(struct device *dev, void *cpu_addr, size_t size,
                      int direction)

Maps a piece of processor virtual memory so it can be accessed by the
device and returns the physical handle of the memory.

The direction for both api's may be converted freely by casting.
However the dma_ API uses a strongly typed enumerator for its
direction:

DMA_NONE                = PCI_DMA_NONE                no direction (used for
                                                debugging)
DMA_TO_DEVICE                = PCI_DMA_TODEVICE        data is going from the
                                                memory to the device
DMA_FROM_DEVICE                = PCI_DMA_FROMDEVICE        data is coming from
                                                the device to the
                                                memory
DMA_BIDIRECTIONAL        = PCI_DMA_BIDIRECTIONAL        direction isn't known

Notes:  Not all memory regions in a machine can be mapped by this
API.  Further, regions that appear to be physically contiguous in
kernel virtual space may not be contiguous as physical memory.  Since
this API does not provide any scatter/gather capability, it will fail
if the user tries to map a non physically contiguous piece of memory.
For this reason, it is recommended that memory mapped by this API be
obtained only from sources which guarantee to be physically contiguous
(like kmalloc).

Further, the physical address of the memory must be within the
dma_mask of the device (the dma_mask represents a bit mask of the
addressable region for the device.  i.e. if the physical address of
the memory anded with the dma_mask is still equal to the physical
address, then the device can perform DMA to the memory).  In order to
ensure that the memory allocated by kmalloc is within the dma_mask,
the driver may specify various platform dependent flags to restrict
the physical memory range of the allocation (e.g. on x86, GFP_DMA
guarantees to be within the first 16Mb of available physical memory,
as required by ISA devices).

Note also that the above constraints on physical contiguity and
dma_mask may not apply if the platform has an IOMMU (a device which
supplies a physical to virtual mapping between the I/O memory bus and
the device).  However, to be portable, device driver writers may *not*
assume that such an IOMMU exists.

Warnings:  Memory coherency operates at a granularity called the cache
line width.  In order for memory mapped by this API to operate
correctly, the mapped region must begin exactly on a cache line
boundary and end exactly on one (to prevent two separately mapped
regions from sharing a single cache line).  Since the cache line size
may not be known at compile time, the API will not enforce this
requirement.  Therefore, it is recommended that driver writers who
don't take special care to determine the cache line size at run time
only map virtual regions that begin and end on page boundaries (which
are guaranteed also to be cache line boundaries).

DMA_TO_DEVICE synchronisation must be done after the last modification
of the memory region by the software and before it is handed off to
the driver.  Once this primitive is used.  Memory covered by this
primitive should be treated as read only by the device.  If the device
may write to it at any point, it should be DMA_BIDIRECTIONAL (see
below).

DMA_FROM_DEVICE synchronisation must be done before the driver
accesses data that may be changed by the device.  This memory should
be treated as read only by the driver.  If the driver needs to write
to it at any point, it should be DMA_BIDIRECTIONAL (see below).

DMA_BIDIRECTIONAL requires special handling: it means that the driver
isn't sure if the memory was modified before being handed off to the
device and also isn't sure if the device will also modify it.  Thus,
you must always sync bidirectional memory twice: once before the
memory is handed off to the device (to make sure all memory changes
are flushed from the processor) and once before the data may be
accessed after being used by the device (to make sure any processor
cache lines are updated with data that the device may have changed.

void
dma_unmap_single(struct device *dev, dma_addr_t dma_addr, size_t size,
                 enum dma_data_direction direction)
void
pci_unmap_single(struct pci_dev *hwdev, dma_addr_t dma_addr,
                 size_t size, int direction)

Unmaps the region previously mapped.  All the parameters passed in
must be identical to those passed in (and returned) by the mapping
API.

bekars

原帖由 albcamus 于 2007-4-25 10:39 發(fā)表于 4樓  


之前是不是要pci_set_master?

傳送的時候不需要，在網(wǎng)卡驅(qū)動probe和resume的時候設(shè)置一次就可以了。

prc

1. 
   
2.     /* Setup the HW Rx Head and Tail Descriptor Pointers and
   
3.      * the Base and Length of the Rx Descriptor Ring */
   
4.     switch (adapter->num_rx_queues) {
   
5. #ifdef CONFIG_E1000_MQ
   
6.     case 2:
   
7.         rdba = adapter->rx_ring[1].dma;
   
8.         E1000_WRITE_REG(hw, RDBAL1, (rdba & 0x00000000ffffffffULL));
   
9.         E1000_WRITE_REG(hw, RDBAH1, (rdba >> 32));
   
10.         E1000_WRITE_REG(hw, RDLEN1, rdlen);
    
11.         E1000_WRITE_REG(hw, RDH1, 0);
    
12.         E1000_WRITE_REG(hw, RDT1, 0);
    
13.         adapter->rx_ring[1].rdh = E1000_RDH1;
    
14.         adapter->rx_ring[1].rdt = E1000_RDT1;
    
15.         /* Fall Through */
    
16. #endif
    
17.     case 1:
    
18.     default:
    
19.         rdba = adapter->rx_ring[0].dma;
    
20.         E1000_WRITE_REG(hw, RDBAL, (rdba & 0x00000000ffffffffULL));
    
21.         E1000_WRITE_REG(hw, RDBAH, (rdba >> 32));
    
22.         E1000_WRITE_REG(hw, RDLEN, rdlen);
    
23.         E1000_WRITE_REG(hw, RDH, 0);
    
24.         E1000_WRITE_REG(hw, RDT, 0);
    
25.         adapter->rx_ring[0].rdh = E1000_RDH;
    
26.         adapter->rx_ring[0].rdt = E1000_RDT;
    
27.         break;
    
28.     }
    

復(fù)制代碼

1. DMA的3個基本要素是目的地址，源地址，傳輸字節(jié)數(shù)。
2. 對于e1000，或者大部分PCI 網(wǎng)卡，DMA控制器在網(wǎng)卡芯片內(nèi)部，由芯片內(nèi)部的狀態(tài)機(jī)控制，是自動啟動的。
3. 源地址，傳輸字節(jié)數(shù)都由網(wǎng)卡芯片自己控制，網(wǎng)卡只需要一個目的地址，以便將數(shù)據(jù)傳送到指定的內(nèi)存空間。
4. DMA傳輸過程對CPU也是透明的，當(dāng)DMA傳輸完成之后，產(chǎn)生一個中斷通知CPU進(jìn)行處理。
5. i386系統(tǒng)有沒有CPU可以控制的DMA控制器我不是很清楚。不過很多嵌入式自身帶有DMA控制器，但是這個東西不具有通用性。因為其它硬件模塊并不知道這些DMA模塊是怎么工作的，除非整個系統(tǒng)都是自己構(gòu)建的。