- Simplify pci_epf_test_alloc_space() by using pci_epc_get_next_free_bar()
as other similar iterators do (Niklas Cassel)
- Configure endpoint BARs as 64-bit if that's all the hardware supports, in
addition to doing it if the BAR size is larger than 2GB (Niklas Cassel)
- Remove superfluous pci_epf_configure_bar(), since pci_epf_alloc_space()
now contains that functionality (Niklas Cassel)
- Simplify pci_epf_test_set_bar() (Niklas Cassel)
- Clean up pci_epf_test_unbind() to reduce indentation level (Niklas
Cassel)
* pci/endpoint:
PCI: endpoint: pci-epf-test: Clean up pci_epf_test_unbind()
PCI: endpoint: pci-epf-test: Simplify pci_epf_test_set_bar() loop
PCI: endpoint: pci-epf-test: Remove superfluous code
PCI: endpoint: Allocate a 64-bit BAR if that is the only option
PCI: endpoint: pci-epf-test: Simplify pci_epf_test_alloc_space() loop
Clean up pci_epf_test_unbind() by using a continue if we did not allocate
memory for the BAR index. This reduces the indentation level by one.
This makes pci_epf_test_unbind() more similar to pci_epf_test_set_bar().
Link: https://lore.kernel.org/linux-pci/20240320113157.322695-6-cassel@kernel.org
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Simplify the loop in pci_epf_test_set_bar().
If we allocated memory for the BAR, we need to call set_bar() for that
BAR, if we did not allocated memory for that BAR, we need to skip.
It is as simple as that. This also matches the logic in
pci_epf_test_unbind().
A 64-bit BAR will still only be one allocation, with the BAR succeeding
the 64-bit BAR being null.
While at it, remove the misleading comment.
A EPC .set_bar() callback should never change the epf_bar->flags.
(E.g. to set a 64-bit BAR if we requested a 32-bit BAR.)
A .set_bar() callback should do what we request it to do.
If it can't satisfy the request, it should return an error.
If platform has a specific requirement, e.g. that a certain BAR has to
be a 64-bit BAR, then it should specify that by setting the .only_64bit
flag for that specific BAR in epc_features->bar[], such that
pci_epf_alloc_space() will return a epf_bar with the 64-bit flag set.
(Such that .set_bar() will receive a request to set a 64-bit BAR.)
Link: https://lore.kernel.org/linux-pci/20240320113157.322695-5-cassel@kernel.org
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
The only reason why we call pci_epf_configure_bar() is to set
PCI_BASE_ADDRESS_MEM_TYPE_64 in case the hardware requires it.
However, this flag is now automatically set when allocating a BAR that
can only be a 64-bit BAR, so we can drop pci_epf_configure_bar()
completely.
Link: https://lore.kernel.org/linux-pci/20240320113157.322695-4-cassel@kernel.org
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Make pci-epf-test use pci_epc_get_next_free_bar() just like pci-epf-ntb.c
and pci-epf-vntb.c.
Using pci_epc_get_next_free_bar() also makes it more obvious that
pci-epf-test does no special configuration at all.
(The only configuration pci-epf-test does is setting
PCI_BASE_ADDRESS_MEM_TYPE_64 if epc_features has marked the specific BAR
as only_64bit. pci_epc_get_next_free_bar() already takes only_64bit into
account when looping.)
This way, the code is more consistent between EPF drivers, and pci-epf-test
does not need to explicitly check if the BAR is reserved, or if the index
belongs to a BAR succeeding a 64-bit only BAR.
Link: https://lore.kernel.org/linux-pci/20240320113157.322695-2-cassel@kernel.org
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
"core_init_notifier" flag is set by the glue drivers requiring refclk from
the host to complete the DWC core initialization. Also, those drivers will
send a notification to the EPF drivers once the initialization is fully
completed using the pci_epc_init_notify() API. Only then, the EPF drivers
will start functioning.
For the rest of the drivers generating refclk locally, EPF drivers will
start functioning post binding with them. EPF drivers rely on the
'core_init_notifier' flag to differentiate between the drivers.
Unfortunately, this creates two different flows for the EPF drivers.
So to avoid that, let's get rid of the "core_init_notifier" flag and follow
a single initialization flow for the EPF drivers. This is done by calling
the dw_pcie_ep_init_notify() from all glue drivers after the completion of
dw_pcie_ep_init_registers() API. This will allow all the glue drivers to
send the notification to the EPF drivers once the initialization is fully
completed.
Only difference here is that, the drivers requiring refclk from host will
send the notification once refclk is received, while others will send it
during probe time itself.
But this also requires the EPC core driver to deliver the notification
after EPF driver bind. Because, the glue driver can send the notification
before the EPF drivers bind() and in those cases the EPF drivers will miss
the event. To accommodate this, EPC core is now caching the state of the
EPC initialization in 'init_complete' flag and pci-ep-cfs driver sends the
notification to EPF drivers based on that after each EPF driver bind.
Link: https://lore.kernel.org/linux-pci/20240327-pci-dbi-rework-v12-8-082625472414@linaro.org
Tested-by: Niklas Cassel <cassel@kernel.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Niklas Cassel <cassel@kernel.org>
The hardware description for BARs is scattered in many different variables
in pci_epc_features. Some of these things are mutually exclusive, so it
can create confusion over which variable that has precedence over another.
Improve the situation by creating a struct pci_epc_bar_desc, and a new
enum pci_epc_bar_type, and convert the endpoint controller drivers to use
this more well defined format.
Additionally, some endpoint controller drivers mark the BAR succeeding a
"64-bit only BAR" as reserved, while some do not. By definition, a 64-bit
BAR uses the succeeding BAR for the upper 32-bits, so an EPF driver cannot
use a BAR succeeding a 64-bit BAR. Ensure that all endpoint controller
drivers are uniform, and actually describe a reserved BAR as reserved.
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Link: https://lore.kernel.org/r/20240216134524.1142149-2-cassel@kernel.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Now that the checks are performed by the pci_epf_alloc_space() API, let's
remove the superfluous checks in this driver.
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20240207213922.1796533-4-cassel@kernel.org
[mani: reworded commit message]
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Refactor pci_epf_alloc_space() API to accept "epc_features" as a parameter.
This is a preparatory work to make the API more robust.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Link: https://lore.kernel.org/r/20240207213922.1796533-2-cassel@kernel.org
[mani: reworded commit message]
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
The pci_epf_ops struct for the PCI endpoint test driver is never modified.
Mark it as const so it can be placed in the read-only section.
[kwilczynski: commit log]
Link: https://lore.kernel.org/linux-pci/20230722230848.589428-5-lars@metafoo.de
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
In the endpoint test function driver, rename IRQ_TYPE_LEGACY to
IRQ_TYPE_INTX and COMMAND_RAISE_LEGACY_IRQ to COMMAND_RAISE_INTX_IRQ
to match the term used in the PCI specifications.
Link: https://lore.kernel.org/r/20231122060406.14695-5-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
linux/pci.h defines the IRQ flags PCI_IRQ_INTX, PCI_IRQ_MSI and
PCI_IRQ_MSIX. Let's use these flags directly instead of the endpoint
definitions provided by enum pci_epc_irq_type. This removes the need
for defining this enum type completely.
Link: https://lore.kernel.org/r/20231122060406.14695-3-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Currently, the EPF probe function doesn't get the device ID argument needed
to correctly identify the device table ID of the EPF device.
When multiple entries are added to the "struct pci_epf_device_id" table,
the probe function needs to identify the correct one. This is achieved by
modifying the pci_epf_match_id() function to return the match ID pointer
and passing it to the driver's probe function.
pci_epf_device_match() function can return bool based on the return value
of pci_epf_match_id().
Link: https://lore.kernel.org/r/20230602114756.36586-3-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
In pci_epf_test_init_dma_chan() epf_test->dma_chan_rx is assigned from
dma_request_channel() with DMA_DEV_TO_MEM as filter.dma_mask.
However, in pci_epf_test_data_transfer() if the dir is DMA_DEV_TO_MEM,
epf->dma_chan_rx should be used but instead we are using
epf_test->dma_chan_tx.
Fix it.
Link: https://lore.kernel.org/r/20230412063447.2841177-1-yoshihiro.shimoda.uh@renesas.com
Fixes: 8353813c88 ("PCI: endpoint: Enable DMA tests for endpoints with DMA capabilities")
Tested-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
In pci_epf_test_print_rate(), instead of open coding a reduction loop to
allow for a division by a 32-bits ns value, simply use div64_u64() to
calculate the transfer rate. To match the printed unit of KB/s, this
calculation divides the rate by 1000 instead of 1024 (that would be KiB/s
unit).
Change the format of the results printed by pci_epf_test_print_rate() to be
more compact without the double new line. Also use dev_info() instead of
pr_info().
Link: https://lore.kernel.org/r/20230415023542.77601-14-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
There is no need to have each read, write and copy test functions check
for the FLAG_USE_DMA flag against the DMA support status indicated by
epf_test->dma_supported. Move this test to the command handler function
pci_epf_test_cmd_handler() to check once for all cases.
Link: https://lore.kernel.org/r/20230415023542.77601-13-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Each of the test functions pci_epf_test_write(), pci_epf_test_read() and
pci_epf_test_copy() return an int result which is used by
pci_epf_test_cmd_handler() to set a success or error bit in the request
status.
In the spirit of keeping the processing of each test case self-contained
within its own test function, move the request status field update from
pci_epf_test_cmd_handler() to each of these test functions and change these
functions declaration to returning void.
Link: https://lore.kernel.org/r/20230415023542.77601-12-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Command codes are never combined together as flags into a single value.
Thus we can replace the series of "if" tests in pci_epf_test_cmd_handler()
with a cleaner switch-case statement. This also allows checking that we
got a valid command and print an error message if we did not.
Link: https://lore.kernel.org/r/20230415023542.77601-11-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
The pci-epf-test driver uses the test register BAR memory directly to get
and execute a test registers set by the RC side and defined using a struct
pci_epf_test_reg. This direct use relies on using the register BAR address
as a pointer to a struct pci_epf_test_reg to execute the test case and to
send back the test result through the status field of struct
pci_epf_test_reg. In practice, the status field is always updated before an
interrupt is raised in pci_epf_test_raise_irq(), to ensure that the RC side
sees the updated status when receiving an interrupt.
However, such assignment direct access does not ensure that changes to the
status register make it to memory, and so visible to the host, before an
interrupt is raised, thus potentially resulting in the RC host not seeing
the correct status result for a test.
Avoid this potential problem by using READ_ONCE()/WRITE_ONCE() when
accessing the command and status fields of a pci_epf_test_reg structure.
This ensure that a test start (pci_epf_test_cmd_handler() function) and
completion (with the function pci_epf_test_raise_irq()) achieve a correct
synchronization with the MMIO register accesses on the RC host.
Link: https://lore.kernel.org/r/20230415023542.77601-10-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
For the commands COMMAND_RAISE_LEGACY_IRQ, COMMAND_RAISE_MSI_IRQ and
COMMAND_RAISE_MSIX_IRQ, the function pci_epf_test_cmd_handler()
sets the STATUS_IRQ_RAISED status flag and calls the epc function
pci_epc_raise_irq() directly. However, this is also exactly what the
pci_epf_test_raise_irq() function does. Avoid duplicating these
operations by directly using pci_epf_test_raise_irq() for the IRQ test
commands. It is OK to do so as the host side endpoint test driver always
set the correct IRQ type for the IRQ test commands.
At the same time, move the IRQ number check done for the
COMMAND_RAISE_MSI_IRQ and COMMAND_RAISE_MSIX_IRQ commands
to pci_epf_test_raise_irq(), to also check the IRQ number requested
by the host for other test commands.
This significantly simplifies pci_epf_test_cmd_handler().
Link: https://lore.kernel.org/r/20230415023542.77601-9-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Change the interface of the function pci_epf_test_raise_irq() to directly
pass a pointer to the struct pci_epf_test_reg defining the test being
executed. This avoids the need for grabbing this pointer using the register
BAR address and simplifies the call sites as the IRQ type and IRQ numbers
do not have to be passed as arguments.
Link: https://lore.kernel.org/r/20230415023542.77601-8-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
The function pci_epf_test_cmd_handler() uses the register BAR address as a
pointer to a struct pci_epf_test_reg to determine the command sent by the
host and to execute the test function accordingly. There is no need for
doing this assignment again in each of the read, write and copy test
functions. We can simply pass the reg pointer as an argument to the
functions pci_epf_test_write(), pci_epf_test_read() and
pci_epf_test_copy().
Link: https://lore.kernel.org/r/20230415023542.77601-7-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Instead of an open coded call to the tx_submit() operation of struct
dma_async_tx_descriptor, use the helper function dmaengine_submit().
No functional change is introduced with this.
Link: https://lore.kernel.org/r/20230415023542.77601-6-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
pci_epf_test_data_transfer() and pci_epf_test_dma_callback() are not
handling DMA transfer completion correctly, leading to completion
notifications to the RC side that are too early. This problem can be
detected when the RC side is running an IOMMU with messages such as:
pci-endpoint-test 0000:0b:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x001c address=0xfff00000 flags=0x0000]
When running the pcitest.sh tests: the address used for a previous
test transfer generates the above error while the next test transfer is
running.
Fix this by testing the DMA transfer status in pci_epf_test_dma_callback()
and notifying the completion only when the transfer status is DMA_COMPLETE
or DMA_ERROR. Furthermore, in pci_epf_test_data_transfer(), be paranoid and
check again the transfer status and always call dmaengine_terminate_sync()
before returning.
Link: https://lore.kernel.org/r/20230415023542.77601-5-dlemoal@kernel.org
Fixes: 8353813c88 ("PCI: endpoint: Enable DMA tests for endpoints with DMA capabilities")
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Cc: stable@vger.kernel.org
Reinitialize the transfer_complete DMA transfer completion before calling
tx_submit(), to avoid seeing the DMA transfer complete before the
completion is initialized, thus potentially losing the completion
notification.
Link: https://lore.kernel.org/r/20230415023542.77601-4-dlemoal@kernel.org
Fixes: 8353813c88 ("PCI: endpoint: Enable DMA tests for endpoints with DMA capabilities")
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Cc: stable@vger.kernel.org
As a part of the transition towards callback mechanism for signalling the
events from EPC to EPF, let's use the link_up() callback in the place of
the LINK_UP notifier. This also removes the notifier support completely
from the PCI endpoint framework.
Link: https://lore.kernel.org/linux-pci/20230124071158.5503-6-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Acked-by: Kishon Vijay Abraham I <kishon@kernel.org>
Instead of using the notifiers for passing the events from EPC to EPF,
let's introduce a callback based mechanism where the EPF drivers can
populate relevant callbacks for EPC events they want to subscribe.
The use of notifiers in kernel is not recommended if there is a real link
between the sender and receiver, like in this case. Also, the existing
atomic notifier forces the notification functions to be in atomic context
while the caller may be in non-atomic context. For instance, the two
in-kernel users of the notifiers, pcie-qcom and pcie-tegra194, both are
calling the notifier functions in non-atomic context (from threaded IRQ
handlers). This creates a sleeping in atomic context issue with the
existing EPF_TEST driver that calls the EPC APIs that may sleep.
For all these reasons, let's get rid of the notifier chains and use the
simple callback mechanism for signalling the events from EPC to EPF
drivers. This preserves the context of the caller and avoids the latency
of going through a separate interface for triggering the notifications.
As a first step of the transition, the core_init() callback is introduced
in this commit, that'll replace the existing CORE_INIT notifier used for
signalling the init complete event from EPC.
During the occurrence of the event, EPC will go over the list of EPF
drivers attached to it and will call the core_init() callback if available.
Link: https://lore.kernel.org/linux-pci/20230124071158.5503-5-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Acked-by: Kishon Vijay Abraham I <kishon@kernel.org>
The pci_epf_test_notifier function should be installed also if only
core_init_notifier is enabled. Fix the current logic.
Link: https://lore.kernel.org/r/20220825090101.20474-1-hayashi.kunihiko@socionext.com
Fixes: 5e50ee27d4 ("PCI: pci-epf-test: Add support to defer core initialization")
Signed-off-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Acked-by: Om Prakash Singh <omp@nvidia.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
- Remove unused struct dw_edma_chip.irq (Frank Li)
- Move eDMA private data from struct dw_edma to struct dw_edma_chip (Frank
Li)
- Convert "struct dw_edma_region rg_region" to "void __iomem *reg_base"
since only the virtual address (not physical address or size) is used
(Frank Li)
- Rename "*_ch_cnt" to "ll_*_cnt" to reflect actual usage (Frank Li)
- Drop dma_slave_config.direction field usage (Serge Semin)
- Fix eDMA Rd/Wr-channels and DMA-direction semantics (Serge Semin)
- Add chip-specific DW_EDMA_CHIP_LOCAL flag to indicate that local eDMA
doesn't require generating MSIs to remote (Frank Li)
- Enable DMA tests for endpoints that support it (Frank Li)
* pci/ctrl/dwc-edma:
PCI: endpoint: Enable DMA tests for endpoints with DMA capabilities
dmaengine: dw-edma: Add support for chip-specific flags
dmaengine: dw-edma: Fix eDMA Rd/Wr-channels and DMA-direction semantics
dmaengine: dw-edma: Drop dma_slave_config.direction field usage
dmaengine: dw-edma: Rename wr(rd)_ch_cnt to ll_wr(rd)_cnt in struct dw_edma_chip
dmaengine: dw-edma: Change rg_region to reg_base in struct dw_edma_chip
dmaengine: dw-edma: Detach the private data and chip info structures
dmaengine: dw-edma: Remove unused irq field in struct dw_edma_chip
Some PCI Endpoint controllers integrate an eDMA (embedded DMA). eDMA can
bypass the outbound memory address translation unit to access all RC memory
space.
Add eDMA support for pci-epf-test.
Depending on HW availability, the EPF test can use either eDMA or general
system DMA controllers to perform DMA. The test tries to use eDMA first and
falls back to general system DMA controllers if there's no eDMA
Separate dma_chan to dma_chan_tx and dma_chan_rx. Search for an eDMA
channel first, then search for a memory-to-memory DMA channel. If general
memory to memory channels are used, dma_chan_rx = dma_chan_tx.
Add dma_addr_t dma_remote in pci_epf_test_data_transfer() because eDMA uses
remote RC physical address directly.
Add enum dma_transfer_direction dir in pci_epf_test_data_transfer() because
eDMA chooses the correct RX/TX channel by dir.
The overall steps are:
1. Execute dma_request_channel() and filter function to find correct eDMA
RX and TX Channel. If a channel does not exist, fallback to try to
allocate general memory to memory DMA channel.
2. Execute dmaengine_slave_config() to configure remote side physical
address.
3. Execute dmaengine_prep_slave_single() to create transfer descriptor.
4. Execute tx_submit().
5. Execute dma_async_issue_pending()
[bhelgaas: squash in fix from Dan Carpenter <dan.carpenter@oracle.com>:
https://lore.kernel.org/r/Ys2GSTnZhuLzzQG5@kili, also previously posted by
Peng Wu <wupeng58@huawei.com>:
https://lore.kernel.org/all/CANXvt5rK98-cEMgpzopY9POOK8a5=VDib8uKPLgJakOG=hRfwQ@mail.gmail.com/]
Link: https://lore.kernel.org/r/20220524152159.2370739-9-Frank.Li@nxp.com
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Acked-By: Vinod Koul <vkoul@kernel.org>
Unbinding an endpoint function from the endpoint controller shouldn't stop
the controller. This is especially a problem for multi-function endpoints
where other endpoints may still be active.
Don't stop the controller when unbinding one of its endpoints. Normally
the controller is stopped via configfs.
Fixes: 349e7a85b2 ("PCI: endpoint: functions: Add an EP function to test PCI")
Link: https://lore.kernel.org/r/20220622040924.113279-1-mie@igel.co.jp
Signed-off-by: Shunsuke Mie <mie@igel.co.jp>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Add virtual function number in pci_epc ops. EPC controller driver
can perform virtual function specific initialization based on the
virtual function number.
Link: https://lore.kernel.org/r/20210819123343.1951-5-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Add the missing destroy_workqueue() before return from
pci_epf_test_init() in the error handling case and add
destroy_workqueue() in pci_epf_test_exit().
Link: https://lore.kernel.org/r/20210331084012.2091010-1-yangyingliang@huawei.com
Fixes: 349e7a85b2 ("PCI: endpoint: functions: Add an EP function to test PCI")
Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
get_features ops of pci_epc_ops may return NULL, causing NULL pointer
dereference in pci_epf_test_alloc_space function. Let us add a check for
pci_epc_feature pointer in pci_epf_test_bind before we access it to avoid
any such NULL pointer dereference and return -ENOTSUPP in case
pci_epc_feature is not found.
When the patch is not applied and EPC features is not implemented in the
platform driver, we see the following dump due to kernel NULL pointer
dereference.
Call trace:
pci_epf_test_bind+0xf4/0x388
pci_epf_bind+0x3c/0x80
pci_epc_epf_link+0xa8/0xcc
configfs_symlink+0x1a4/0x48c
vfs_symlink+0x104/0x184
do_symlinkat+0x80/0xd4
__arm64_sys_symlinkat+0x1c/0x24
el0_svc_common.constprop.3+0xb8/0x170
el0_svc_handler+0x70/0x88
el0_svc+0x8/0x640
Code: d2800581 b9403ab9 f9404ebb 8b394f60 (f9400400)
---[ end trace a438e3c5a24f9df0 ]---
Link: https://lore.kernel.org/r/20210324101609.79278-1-shradha.t@samsung.com
Fixes: 2c04c5b8ee ("PCI: pci-epf-test: Use pci_epc_get_features() to get EPC features")
Signed-off-by: Sriram Dash <dash.sriram@gmail.com>
Signed-off-by: Shradha Todi <shradha.t@samsung.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Pankaj Dubey <pankaj.dubey@samsung.com>
Reviewed-by: Kishon Vijay Abraham I <kishon@ti.com>
In the case of standard endpoint functions, only one endpoint controller
(EPC) will be associated with an endpoint function (EPF). However for
providing NTB (non transparent bridge) functionality, two EPCs should be
associated with a single EPF. Add support to associate secondary EPC with
EPF. This is in preparation for adding NTB endpoint function driver.
Link: https://lore.kernel.org/r/20210201195809.7342-7-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Modify pci_epc_get_next_free_bar() and pci_epc_get_first_free_bar() to
return error values if there are no free BARs available.
Link: https://lore.kernel.org/r/20210201195809.7342-5-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
When unbinding pci_epf_test, pci_epf_test_clean_dma_chan() is called in
pci_epf_test_unbind() even though epf_test->dma_supported is false.
As a result, dma_release_channel() will trigger a NULL pointer
dereference because dma_chan is not set.
Avoid calling dma_release_channel() if epf_test->dma_supported
is false.
Link: https://lore.kernel.org/r/1587540287-10458-1-git-send-email-hayashi.kunihiko@socionext.com
Fixes: 5ebf3fc59b ("PCI: endpoint: functions/pci-epf-test: Add DMA support to transfer data")
Signed-off-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
[lorenzo.pieralisi@arm.com: commit log]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
commit 8963106eab ("PCI: endpoint: Add MSI-X interfaces") while
adding support to raise MSI-X interrupts from endpoint didn't include
BAR Indicator register (BIR) configuration and MSI-X table offset as
arguments in pci_epc_set_msix(). This would result in endpoint
controller register using random BAR indicator register, the memory
for which might not be allocated by the endpoint function driver.
Add BAR indicator register and MSI-X table offset as arguments in
pci_epc_set_msix() and allocate space for MSI-X table and pending
bit array (PBA) in pci-epf-test endpoint function driver.
Fixes: 8963106eab ("PCI: endpoint: Add MSI-X interfaces")
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Print throughput information in KB/s after every completed transfer,
including information on whether DMA is used or not.
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Alan Mikhak <alan.mikhak@sifive.com>
Use dmaengine API and add support for transferring data using DMA.
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Alan Mikhak <alan.mikhak@sifive.com>
Add support to defer core initialization for the endpoint mode of
operation.
This would enable support for implementations where the core
initialization needs to be deferred until the PCIe reference clock is
available from the host system.
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
[lorenzo.pieralisi@arm.com: commit log]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Use atomic_notifier_call_chain() to notify EPC events like linkup to EPF
driver instead of using linkup ops in EPF driver. This is in preparation
for adding proper locking mechanism to EPF ops. This will also enable to
add more events (in addition to linkup) in the future.
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Vidya Sagar <vidyas@nvidia.com>
Code that iterates over all standard PCI BARs typically uses
PCI_STD_RESOURCE_END. However, that requires the unusual test
"i <= PCI_STD_RESOURCE_END" rather than something the typical
"i < PCI_STD_NUM_BARS".
Add a definition for PCI_STD_NUM_BARS and change loops to use the more
idiomatic C style to help avoid fencepost errors.
Link: https://lore.kernel.org/r/20190927234026.23342-1-efremov@linux.com
Link: https://lore.kernel.org/r/20190927234308.23935-1-efremov@linux.com
Link: https://lore.kernel.org/r/20190916204158.6889-3-efremov@linux.com
Signed-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Sebastian Ott <sebott@linux.ibm.com> # arch/s390/
Acked-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com> # video/fbdev/
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com> # pci/controller/dwc/
Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com> # scsi/pm8001/
Acked-by: Martin K. Petersen <martin.petersen@oracle.com> # scsi/pm8001/
Acked-by: Ulf Hansson <ulf.hansson@linaro.org> # memstick/
Associated pci_epf_bar structure is needed in pci_epc_clear_bar() to
clear a BAR correctly but it is reset in pci_epf_free_space() (that
is called first) which results in pci_epc_clear_bar() failure.
Reorder the pci_epc_clear_bar()/pci_epf_free_space() calls execution
to fix the issue.
Signed-off-by: Alan Mikhak <alan.mikhak@sifive.com>
[lorenzo.pieralisi@arm.com: reworded the commit log]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
