aboutsummaryrefslogtreecommitdiffstats
path: root/crypto
Commit message (Collapse)AuthorAgeFilesLines
...
| * crypto: aead - Fix checkpatch errorsRichard Hartmann2010-02-161-2/+2
| | | | | | | | | | Signed-off-by: Richard Hartmann <richih.mailinglist@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: ablkcipher - Fix checkpatch errorsRichard Hartmann2010-02-161-2/+2
| | | | | | | | | | Signed-off-by: Richard Hartmann <richih.mailinglist@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: pcrypt - call the complete function on errorSteffen Klassert2010-02-041-3/+3
| | | | | | | | | | | | | | | | This fixes three forgotten calls to the complete function in the error case. Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: md5 - Add export supportMax Vozeler2010-01-171-16/+24
| | | | | | | | | | | | | | | | | | | | This patch adds export/import support to md5. The exported type is defined by struct md5_state. This is modeled after the equivalent change to sha1_generic. Signed-off-by: Max Vozeler <max@hinterhof.net> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: gcm - Add RFC4543 wrapper for GCMTobias Brunner2010-01-171-0/+287
| | | | | | | | | | | | | | | | | | This patch adds the RFC4543 (GMAC) wrapper for GCM similar to the existing RFC4106 wrapper. The main differences between GCM and GMAC are the contents of the AAD and that the plaintext is empty for the latter. Signed-off-by: Tobias Brunner <tobias@strongswan.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: pcrypt - Add pcrypt crypto parallelization wrapperSteffen Klassert2010-01-073-0/+456
| | | | | | | | | | | | | | | | | | This patch adds a parallel crypto template that takes a crypto algorithm and converts it to process the crypto transforms in parallel. For the moment only aead algorithms are supported. Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * crypto: testmgr - Fix complain about lack test for internal used algorithmYouquan, Song2009-12-231-0/+84
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When load aesni-intel and ghash_clmulni-intel driver,kernel will complain no test for some internal used algorithm. The strange information as following: alg: No test for __aes-aesni (__driver-aes-aesni) alg: No test for __ecb-aes-aesni (__driver-ecb-aes-aesni) alg: No test for __cbc-aes-aesni (__driver-cbc-aes-aesni) alg: No test for __ecb-aes-aesni (cryptd(__driver-ecb-aes-aesni) alg: No test for __ghash (__ghash-pclmulqdqni) alg: No test for __ghash (cryptd(__ghash-pclmulqdqni)) This patch add NULL test entries for these algorithm and driver. Signed-off-by: Youquan, Song <youquan.song@intel.com> Signed-off-by: Ying, Huang <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | Merge branch 'fixes' of ↵Linus Torvalds2009-12-301-0/+7
|\ \ | |/ |/| | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx * 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx: drivers/dma: Correct use after free drivers/dma: drop unnecesary memset ioat2,3: put channel hardware in known state at init async_tx: expand async raid6 test to cover ioatdma corner case ioat3: fix p-disabled q-continuation sh: fix DMA driver's descriptor chaining and cookie assignment dma: at_hdmac: correct incompatible type for argument 1 of 'spin_lock_bh'
| * async_tx: expand async raid6 test to cover ioatdma corner caseDan Williams2009-12-171-0/+7
| | | | | | | | | | | | | | Add explicit 11 and 12 disks cases to exercise the 0 < src_cnt % 8 < 3 corner case in the ioatdma driver. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* | Merge branch 'for-linus' of ↵Linus Torvalds2009-12-141-1/+1
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu: (34 commits) m68k: rename global variable vmalloc_end to m68k_vmalloc_end percpu: add missing per_cpu_ptr_to_phys() definition for UP percpu: Fix kdump failure if booted with percpu_alloc=page percpu: make misc percpu symbols unique percpu: make percpu symbols in ia64 unique percpu: make percpu symbols in powerpc unique percpu: make percpu symbols in x86 unique percpu: make percpu symbols in xen unique percpu: make percpu symbols in cpufreq unique percpu: make percpu symbols in oprofile unique percpu: make percpu symbols in tracer unique percpu: make percpu symbols under kernel/ and mm/ unique percpu: remove some sparse warnings percpu: make alloc_percpu() handle array types vmalloc: fix use of non-existent percpu variable in put_cpu_var() this_cpu: Use this_cpu_xx in trace_functions_graph.c this_cpu: Use this_cpu_xx for ftrace this_cpu: Use this_cpu_xx in nmi handling this_cpu: Use this_cpu operations in RCU this_cpu: Use this_cpu ops for VM statistics ... Fix up trivial (famous last words) global per-cpu naming conflicts in arch/x86/kvm/svm.c mm/slab.c
| * | this_cpu: Use this_cpu_ptr in crypto subsystemChristoph Lameter2009-10-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Just a slight optimization that removes one array lookup. The processor number is needed for other things as well so the get/put_cpu cannot be removed. Acked-by: Tejun Heo <tj@kernel.org> Cc: Huang Ying <ying.huang@intel.com> Signed-off-by: Christoph Lameter <cl@linux-foundation.org> Signed-off-by: Tejun Heo <tj@kernel.org>
* | | Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6Linus Torvalds2009-12-088-443/+111
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: hwrng: core - Prevent too-small buffer sizes hwrng: virtio-rng - Convert to new API hwrng: core - Replace u32 in driver API with byte array crypto: ansi_cprng - Move FIPS functions under CONFIG_CRYPTO_FIPS crypto: testmgr - Add ghash algorithm test before provide to users crypto: ghash-clmulni-intel - Put proper .data section in place crypto: ghash-clmulni-intel - Use gas macro for PCLMULQDQ-NI and PSHUFB crypto: aesni-intel - Use gas macro for AES-NI instructions x86: Generate .byte code for some new instructions via gas macro crypto: ghash-intel - Fix irq_fpu_usable usage crypto: ghash-intel - Add PSHUFB macros crypto: ghash-intel - Hard-code pshufb crypto: ghash-intel - Fix building failure on x86_32 crypto: testmgr - Fix warning crypto: ansi_cprng - Fix test in get_prng_bytes crypto: hash - Remove cra_u.{digest,hash} crypto: api - Remove digest case from procfs show handler crypto: hash - Remove legacy hash/digest code crypto: ansi_cprng - Add FIPS wrapper crypto: ghash - Add PCLMULQDQ accelerated implementation
| * \ \ Merge git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6Herbert Xu2009-12-0111-247/+1445
| |\ \ \ | | | |/ | | |/|
| * | | crypto: ansi_cprng - Move FIPS functions under CONFIG_CRYPTO_FIPSJaswinder Singh Rajput2009-11-231-28/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fips_cprng_get_random and fips_cprng_reset is used only by CONFIG_CRYPTO_FIPS. This also fixes compilation warnings: crypto/ansi_cprng.c:360: warning: ‘fips_cprng_get_random’ defined but not used crypto/ansi_cprng.c:393: warning: ‘fips_cprng_reset’ defined but not used Signed-off-by: Jaswinder Singh Rajput <jaswinderrajput@gmail.com> Acked-by: Neil Horman <nhorman@tuxdriver.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * | | crypto: testmgr - Add ghash algorithm test before provide to usersYouquan, Song2009-11-232-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add ghash algorithm test before provide it to users Signed-off-by: Youquan, Song <youquan.song@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * | | crypto: ghash-intel - Fix building failure on x86_32Huang Ying2009-10-271-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | CLMUL-NI accelerated GHASH should be turned off on non-x86_64 machine. Reported-by: Dave Young <hidave.darkstar@gmail.com> Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * | | crypto: testmgr - Fix warningFelipe Contreras2009-10-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | crypto/testmgr.c: In function ‘test_cprng’: crypto/testmgr.c:1204: warning: ‘err’ may be used uninitialized in this function Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * | | crypto: ansi_cprng - Fix test in get_prng_bytesRoel Kluin2009-10-271-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | size_t nbytes cannot be less than 0 and the test was redundant. Signed-off-by: Roel Kluin <roel.kluin@gmail.com> Acked-by: Neil Horman <nhorman@tuxdriver.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * | | crypto: api - Remove digest case from procfs show handlerBenjamin Gilbert2009-10-191-7/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove special handling of old-style digest algorithms from the procfs show handler. Signed-off-by: Benjamin Gilbert <bgilbert@cs.cmu.edu> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * | | crypto: hash - Remove legacy hash/digest codeBenjamin Gilbert2009-10-192-423/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 6941c3a0 disabled compilation of the legacy digest code but didn't actually remove it. Rectify this. Also, remove the crypto_hash_type extern declaration from algapi.h now that the struct is gone. Signed-off-by: Benjamin Gilbert <bgilbert@cs.cmu.edu> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * | | crypto: ansi_cprng - Add FIPS wrapperNeil Horman2009-10-191-9/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Patch to add fips(ansi_cprng) alg, which is ansi_cprng plus a continuous test Signed-off-by: Neil Horman <nhorman@tuxdriver.com> Acked-by: Jarod Wilson <jarod@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
| * | | crypto: ghash - Add PCLMULQDQ accelerated implementationHuang Ying2009-10-192-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PCLMULQDQ is used to accelerate the most time-consuming part of GHASH, carry-less multiplication. More information about PCLMULQDQ can be found at: http://software.intel.com/en-us/articles/carry-less-multiplication-and-its-usage-for-computing-the-gcm-mode/ Because PCLMULQDQ changes XMM state, its usage must be enclosed with kernel_fpu_begin/end, which can be used only in process context, the acceleration is implemented as crypto_ahash. That is, request in soft IRQ context will be defered to the cryptd kernel thread. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | | | Merge git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/sysctl-2.6Linus Torvalds2009-12-081-9/+3
|\ \ \ \ | |_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/sysctl-2.6: (43 commits) security/tomoyo: Remove now unnecessary handling of security_sysctl. security/tomoyo: Add a special case to handle accesses through the internal proc mount. sysctl: Drop & in front of every proc_handler. sysctl: Remove CTL_NONE and CTL_UNNUMBERED sysctl: kill dead ctl_handler definitions. sysctl: Remove the last of the generic binary sysctl support sysctl net: Remove unused binary sysctl code sysctl security/tomoyo: Don't look at ctl_name sysctl arm: Remove binary sysctl support sysctl x86: Remove dead binary sysctl support sysctl sh: Remove dead binary sysctl support sysctl powerpc: Remove dead binary sysctl support sysctl ia64: Remove dead binary sysctl support sysctl s390: Remove dead sysctl binary support sysctl frv: Remove dead binary sysctl support sysctl mips/lasat: Remove dead binary sysctl support sysctl drivers: Remove dead binary sysctl support sysctl crypto: Remove dead binary sysctl support sysctl security/keys: Remove dead binary sysctl support sysctl kernel: Remove binary sysctl logic ...
| * | | sysctl: Drop & in front of every proc_handler.Eric W. Biederman2009-11-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For consistency drop & in front of every proc_handler. Explicity taking the address is unnecessary and it prevents optimizations like stubbing the proc_handlers to NULL. Cc: Alexey Dobriyan <adobriyan@gmail.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Joe Perches <joe@perches.com> Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
| * | | sysctl crypto: Remove dead binary sysctl supportEric W. Biederman2009-11-121-8/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that sys_sysctl is a generic wrapper around /proc/sys .ctl_name and .strategy members of sysctl tables are dead code. Remove them. Acked-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
* | | | Merge git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6Linus Torvalds2009-11-301-34/+73
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | * git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: crypto: gcm - fix another complete call in complete fuction crypto: padlock-aes - Use the correct mask when checking whether copying is required
| * | | | crypto: gcm - fix another complete call in complete fuctionHuang Ying2009-11-161-34/+73
| | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The flow of the complete function (xxx_done) in gcm.c is as follow: void complete(struct crypto_async_request *areq, int err) { struct aead_request *req = areq->data; if (!err) { err = async_next_step(); if (err == -EINPROGRESS || err == -EBUSY) return; } complete_for_next_step(areq, err); } But *areq may be destroyed in async_next_step(), this makes complete_for_next_step() can not work properly. To fix this, one of following methods is used for each complete function. - Add a __complete() for each complete(), which accept struct aead_request *req instead of areq, so avoid using areq after it is destroyed. - Expand complete_for_next_step(). The fixing method is based on the idea of Herbert Xu. Signed-off-by: Huang Ying <ying.huang@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* | | | async_tx: build-time toggling of async_{syndrome,xor}_val dma supportDan Williams2009-11-193-6/+28
| |/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ioat3.2 does not support asynchronous error notifications which makes the driver experience latencies when non-zero pq validate results are expected. Provide a mechanism for turning off async_xor_val and async_syndrome_val via Kconfig. This approach is generally useful for any driver that specifies ASYNC_TX_DISABLE_CHANNEL_SWITCH and would like to force the async_tx api to fall back to the synchronous path for certain operations. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* | | Merge branch 'for-linus' of git://neil.brown.name/mdLinus Torvalds2009-10-313-65/+113
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * 'for-linus' of git://neil.brown.name/md: async_tx: fix asynchronous raid6 recovery for ddf layouts async_pq: rename scribble page async_pq: kill a stray dma_map() call and other cleanups md/raid6: kill a gcc-4.0.1 'uninitialized variable' warning raid6/async_tx: handle holes in block list in async_syndrome_val md/async: don't pass a memory pointer as a page pointer. md: Fix handling of raid5 array which is being reshaped to fewer devices. md: fix problems with RAID6 calculations for DDF. md/raid456: downlevel multicore operations to raid_run_ops md: drivers/md/unroll.pl replaced with awk analog md: remove clumsy usage of do_sync_mapping_range from bitmap code md: raid1/raid10: handle allocation errors during array setup. md/raid5: initialize conf->device_lock earlier md/raid1/raid10: add a cond_resched Revert "md: do not progress the resync process if the stripe was blocked"
| * | | async_tx: fix asynchronous raid6 recovery for ddf layoutsDan Williams2009-10-191-30/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The raid6 recovery code currently requires special handling of the 4-disk and 5-disk recovery scenarios for the native layout. Quoting from commit 0a82a623: In these situations the default N-disk algorithm will present 0-source or 1-source operations to dma devices. To cover for dma devices where the minimum source count is 2 we implement 4-disk and 5-disk handling in the recovery code. The ddf layout presents disks=6 and disks=7 to the recovery code in these situations. Instead of looking at the number of disks count the number of non-zero sources in the list and call the special case code when the number of non-failed sources is 0 or 1. [neilb@suse.de: replace 'ddf' flag with counting good sources] Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * | | async_pq: rename scribble pageDan Williams2009-10-191-7/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The global scribble page is used as a temporary destination buffer when disabling the P or Q result is requested. The local scribble buffer contains memory for performing address conversions. Rename the global variable to avoid confusion. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * | | async_pq: kill a stray dma_map() call and other cleanupsDan Williams2009-10-191-7/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - update the kernel doc for async_syndrome to indicate what NULL in the source list means - whitespace fixups Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * | | raid6/async_tx: handle holes in block list in async_syndrome_valNeilBrown2009-10-162-14/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | async_syndrome_val check the P and Q blocks used for RAID6 calculations. With DDF raid6, some of the data blocks might be NULL, so this needs to be handled in the same way that async_gen_syndrome handles it. As async_syndrome_val calls async_xor, also enhance async_xor to detect and skip NULL blocks in the list. Signed-off-by: NeilBrown <neilb@suse.de>
| * | | md/async: don't pass a memory pointer as a page pointer.NeilBrown2009-10-162-16/+15
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | md/raid6 passes a list of 'struct page *' to the async_tx routines, which then either DMA map them for offload, or take the page_address for CPU based calculations. For RAID6 we sometime leave 'blanks' in the list of pages. For CPU based calcs, we want to treat theses as a page of zeros. For offloaded calculations, we simply don't pass a page to the hardware. Currently the 'blanks' are encoded as a pointer to raid6_empty_zero_page. This is a 4096 byte memory region, not a 'struct page'. This is mostly handled correctly but is rather ugly. So change the code to pass and expect a NULL pointer for the blanks. When taking page_address of a page, we need to check for a NULL and in that case use raid6_empty_zero_page. Signed-off-by: NeilBrown <neilb@suse.de>
* | | headers: remove sched.h from interrupt.hAlexey Dobriyan2009-10-111-0/+1
|/ / | | | | | | | | | | | | | | After m68k's task_thread_info() doesn't refer to current, it's possible to remove sched.h from interrupt.h and not break m68k! Many thanks to Heiko Carstens for allowing this. Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
* | Merge branch 'next' of ↵NeilBrown2009-09-239-204/+1292
|\ \ | |/ |/| | | git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx into for-linus
| * async_tx/raid6: add missing dma_unmap calls to the async fail caseDan Williams2009-09-211-0/+13
| | | | | | | | | | | | | | | | If we are unable to offload async_mult() or async_sum_product(), then unmap the buffers before falling through to the synchronous path. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * raid6test: fix stack overflowDan Williams2009-09-161-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Testing on x86_64 with NDISKS=255 yields: do_IRQ: modprobe near stack overflow (cur:ffff88007d19c000,sp:ffff88007d19c128) ...and eventually general protection fault: 0000 [#1] Moving the scribble buffers off the stack allows the test to complete successfully. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * Merge branch 'dmaengine' into async-tx-nextDan Williams2009-09-0811-234/+1155
| |\ | | | | | | | | | | | | | | | | | | | | | Conflicts: crypto/async_tx/async_xor.c drivers/dma/ioat/dma_v2.h drivers/dma/ioat/pci.c drivers/md/raid5.c
| | * Merge commit 'v2.6.31-rc1' into dmaengineDan Williams2009-09-0811-234/+1155
| | |\
| * | | dmaengine, async_tx: support alignment checksDan Williams2009-09-084-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some engines have transfer size and address alignment restrictions. Add a per-operation alignment property to struct dma_device that the async routines and dmatest can use to check alignment capabilities. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * | | dmaengine, async_tx: add a "no channel switch" allocatorDan Williams2009-09-081-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Channel switching is problematic for some dmaengine drivers as the architecture precludes separating the ->prep from ->submit. In these cases the driver can select ASYNC_TX_DISABLE_CHANNEL_SWITCH to modify the async_tx allocator to only return channels that support all of the required asynchronous operations. For example MD_RAID456=y selects support for asynchronous xor, xor validate, pq, pq validate, and memcpy. When ASYNC_TX_DISABLE_CHANNEL_SWITCH=y any channel with all these capabilities is marked DMA_ASYNC_TX allowing async_tx_find_channel() to quickly locate compatible channels with the guarantee that dependency chains will remain on one channel. When ASYNC_TX_DISABLE_CHANNEL_SWITCH=n async_tx_find_channel() may select channels that lead to operation chains that need to cross channel boundaries using the async_tx channel switch capability. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * | | dmaengine: add fence supportDan Williams2009-09-085-27/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some engines optimize operation by reading ahead in the descriptor chain such that descriptor2 may start execution before descriptor1 completes. If descriptor2 depends on the result from descriptor1 then a fence is required (on descriptor2) to disable this optimization. The async_tx api could implicitly identify dependencies via the 'depend_tx' parameter, but that would constrain cases where the dependency chain only specifies a completion order rather than a data dependency. So, provide an ASYNC_TX_FENCE to explicitly identify data dependencies. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| * | | Merge branch 'md-raid6-accel' into ioat3.2Dan Williams2009-09-089-201/+1247
| |\ \ \ | | | | | | | | | | | | | | | | | | | | Conflicts: include/linux/dmaengine.h
| | * | | async_tx: raid6 recovery self testDan Williams2009-08-292-0/+242
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Port drivers/md/raid6test/test.c to use the async raid6 recovery routines. This is meant as a unit test for raid6 acceleration drivers. In addition to the 16-drive test case this implements tests for the 4-disk and 5-disk special cases (dma devices can not generically handle less than 2 sources), and adds a test for the D+Q case. Reviewed-by: Andre Noll <maan@systemlinux.org> Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| | * | | async_tx: add support for asynchronous RAID6 recovery operationsDan Williams2009-08-293-0/+454
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | async_raid6_2data_recov() recovers two data disk failures async_raid6_datap_recov() recovers a data disk and the P disk These routines are a port of the synchronous versions found in drivers/md/raid6recov.c. The primary difference is breaking out the xor operations into separate calls to async_xor. Two helper routines are introduced to perform scalar multiplication where needed. async_sum_product() multiplies two sources by scalar coefficients and then sums (xor) the result. async_mult() simply multiplies a single source by a scalar. This implemention also includes, in contrast to the original synchronous-only code, special case handling for the 4-disk and 5-disk array cases. In these situations the default N-disk algorithm will present 0-source or 1-source operations to dma devices. To cover for dma devices where the minimum source count is 2 we implement 4-disk and 5-disk handling in the recovery code. [ Impact: asynchronous raid6 recovery routines for 2data and datap cases ] Cc: Yuri Tikhonov <yur@emcraft.com> Cc: Ilya Yanok <yanok@emcraft.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: David Woodhouse <David.Woodhouse@intel.com> Reviewed-by: Andre Noll <maan@systemlinux.org> Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| | * | | async_tx: add support for asynchronous GF multiplicationDan Williams2009-08-294-1/+394
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [ Based on an original patch by Yuri Tikhonov ] This adds support for doing asynchronous GF multiplication by adding two additional functions to the async_tx API: async_gen_syndrome() does simultaneous XOR and Galois field multiplication of sources. async_syndrome_val() validates the given source buffers against known P and Q values. When a request is made to run async_pq against more than the hardware maximum number of supported sources we need to reuse the previous generated P and Q values as sources into the next operation. Care must be taken to remove Q from P' and P from Q'. For example to perform a 5 source pq op with hardware that only supports 4 sources at a time the following approach is taken: p, q = PQ(src0, src1, src2, src3, COEF({01}, {02}, {04}, {08})) p', q' = PQ(p, q, q, src4, COEF({00}, {01}, {00}, {10})) p' = p + q + q + src4 = p + src4 q' = {00}*p + {01}*q + {00}*q + {10}*src4 = q + {10}*src4 Note: 4 is the minimum acceptable maxpq otherwise we punt to synchronous-software path. The DMA_PREP_CONTINUE flag indicates to the driver to reuse p and q as sources (in the above manner) and fill the remaining slots up to maxpq with the new sources/coefficients. Note1: Some devices have native support for P+Q continuation and can skip this extra work. Devices with this capability can advertise it with dma_set_maxpq. It is up to each driver how to handle the DMA_PREP_CONTINUE flag. Note2: The api supports disabling the generation of P when generating Q, this is ignored by the synchronous path but is implemented by some dma devices to save unnecessary writes. In this case the continuation algorithm is simplified to only reuse Q as a source. Cc: H. Peter Anvin <hpa@zytor.com> Cc: David Woodhouse <David.Woodhouse@intel.com> Signed-off-by: Yuri Tikhonov <yur@emcraft.com> Signed-off-by: Ilya Yanok <yanok@emcraft.com> Reviewed-by: Andre Noll <maan@systemlinux.org> Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| | * | | async_tx: remove walk of tx->parent chain in dma_wait_for_async_txDan Williams2009-08-291-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We currently walk the parent chain when waiting for a given tx to complete however this walk may race with the driver cleanup routine. The routines in async_raid6_recov.c may fall back to the synchronous path at any point so we need to be prepared to call async_tx_quiesce() (which calls dma_wait_for_async_tx). To remove the ->parent walk we guarantee that every time a dependency is attached ->issue_pending() is invoked, then we can simply poll the initial descriptor until completion. This also allows for a lighter weight 'issue pending' implementation as there is no longer a requirement to iterate through all the channels' ->issue_pending() routines as long as operations have been submitted in an ordered chain. async_tx_issue_pending() is added for this case. Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| | * | | async_tx: kill needless module_{init|exit}Dan Williams2009-08-293-40/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If module_init and module_exit are nops then neither need to be defined. [ Impact: pure cleanup ] Reviewed-by: Andre Noll <maan@systemlinux.org> Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
| | * | | async_tx: add sum check flagsDan Williams2009-08-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace the flat zero_sum_result with a collection of flags to contain the P (xor) zero-sum result, and the soon to be utilized Q (raid6 reed solomon syndrome) zero-sum result. Use the SUM_CHECK_ namespace instead of DMA_ since these flags will be used on non-dma-zero-sum enabled platforms. Reviewed-by: Andre Noll <maan@systemlinux.org> Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
generated by cgit v1.1 at 2024-12-26 16:34:23 +0000