From a68cf983f635930ea35f9e96b27d96598550dea0 Mon Sep 17 00:00:00 2001 From: Mark Nutter Date: Wed, 4 Oct 2006 17:26:12 +0200 Subject: [POWERPC] spufs: scheduler support for NUMA. This patch adds NUMA support to the the spufs scheduler. The new arch/powerpc/platforms/cell/spufs/sched.c is greatly simplified, in an attempt to reduce complexity while adding support for NUMA scheduler domains. SPUs are allocated starting from the calling thread's node, moving to others as supported by current->cpus_allowed. Preemption is gone as it was buggy, but should be re-enabled in another patch when stable. The new arch/powerpc/platforms/cell/spu_base.c maintains idle lists on a per-node basis, and allows caller to specify which node(s) an SPU should be allocated from, while passing -1 tells spu_alloc() that any node is allowed. Since the patch removes the currently implemented preemptive scheduling, it is technically a regression, but practically all users have since migrated to this version, as it is part of the IBM SDK and the yellowdog distribution, so there is not much point holding it back while the new preemptive scheduling patch gets delayed further. Signed-off-by: Arnd Bergmann Signed-off-by: Paul Mackerras --- include/asm-powerpc/spu.h | 1 + 1 file changed, 1 insertion(+) (limited to 'include') diff --git a/include/asm-powerpc/spu.h b/include/asm-powerpc/spu.h index b42b53c..f6c0a95 100644 --- a/include/asm-powerpc/spu.h +++ b/include/asm-powerpc/spu.h @@ -147,6 +147,7 @@ struct spu { }; struct spu *spu_alloc(void); +struct spu *spu_alloc_node(int node); void spu_free(struct spu *spu); int spu_irq_class_0_bottom(struct spu *spu); int spu_irq_class_1_bottom(struct spu *spu); -- cgit v1.1 From 9add11daeee2f6d69f6b86237f197824332a4a3b Mon Sep 17 00:00:00 2001 From: Arnd Bergmann Date: Wed, 4 Oct 2006 17:26:14 +0200 Subject: [POWERPC] spufs: implement error event delivery to user space This tries to fix spufs so we have an interface closer to what is specified in the man page for events returned in the third argument of spu_run. Fortunately, libspe has never been using the returned contents of that register, as they were the same as the return code of spu_run (duh!). Unlike the specification that we never implemented correctly, we now require a SPU_CREATE_EVENTS_ENABLED flag passed to spu_create, in order to get the new behavior. When this flag is not passed, spu_run will simply ignore the third argument now. Signed-off-by: Arnd Bergmann Signed-off-by: Paul Mackerras --- include/asm-powerpc/spu.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) (limited to 'include') diff --git a/include/asm-powerpc/spu.h b/include/asm-powerpc/spu.h index f6c0a95..87cc21e 100644 --- a/include/asm-powerpc/spu.h +++ b/include/asm-powerpc/spu.h @@ -138,6 +138,7 @@ struct spu { void (* ibox_callback)(struct spu *spu); void (* stop_callback)(struct spu *spu); void (* mfc_callback)(struct spu *spu); + void (* dma_callback)(struct spu *spu, int type); char irq_c0[8]; char irq_c1[8]; @@ -169,6 +170,19 @@ extern struct spufs_calls { struct module *owner; } spufs_calls; +/* return status from spu_run, same as in libspe */ +#define SPE_EVENT_DMA_ALIGNMENT 0x0008 /*A DMA alignment error */ +#define SPE_EVENT_SPE_ERROR 0x0010 /*An illegal instruction error*/ +#define SPE_EVENT_SPE_DATA_SEGMENT 0x0020 /*A DMA segmentation error */ +#define SPE_EVENT_SPE_DATA_STORAGE 0x0040 /*A DMA storage error */ +#define SPE_EVENT_INVALID_DMA 0x0800 /* Invalid MFC DMA */ + +/* + * Flags for sys_spu_create. + */ +#define SPU_CREATE_EVENTS_ENABLED 0x0001 +#define SPU_CREATE_FLAG_ALL 0x0001 /* mask of all valid flags */ + #ifdef CONFIG_SPU_FS_MODULE int register_spu_syscalls(struct spufs_calls *calls); void unregister_spu_syscalls(struct spufs_calls *calls); -- cgit v1.1 From 6263203ed6e9ff107129a1ebe613290b342a4465 Mon Sep 17 00:00:00 2001 From: Arnd Bergmann Date: Wed, 4 Oct 2006 17:26:15 +0200 Subject: [POWERPC] spufs: Add infrastructure needed for gang scheduling Add the concept of a gang to spufs as a new type of object. So far, this has no impact whatsover on scheduling, but makes it possible to add that later. A new type of object in spufs is now a spu_gang. It is created with the spu_create system call with the flags argument set to SPU_CREATE_GANG (0x2). Inside of a spu_gang, it is then possible to create spu_context objects, which until now was only possible at the root of spufs. There is a new member in struct spu_context pointing to the spu_gang it belongs to, if any. The spu_gang maintains a list of spu_context structures that are its children. This information can then be used in the scheduler in the future. There is still a bug that needs to be resolved in this basic infrastructure regarding the order in which objects are removed. When the spu_gang file descriptor is closed before the spu_context descriptors, we leak the dentry and inode for the gang. Any ideas how to cleanly solve this are appreciated. Signed-off-by: Arnd Bergmann Signed-off-by: Paul Mackerras --- include/asm-powerpc/spu.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) (limited to 'include') diff --git a/include/asm-powerpc/spu.h b/include/asm-powerpc/spu.h index 87cc21e..83b6dae 100644 --- a/include/asm-powerpc/spu.h +++ b/include/asm-powerpc/spu.h @@ -181,7 +181,10 @@ extern struct spufs_calls { * Flags for sys_spu_create. */ #define SPU_CREATE_EVENTS_ENABLED 0x0001 -#define SPU_CREATE_FLAG_ALL 0x0001 /* mask of all valid flags */ +#define SPU_CREATE_GANG 0x0002 + +#define SPU_CREATE_FLAG_ALL 0x0003 /* mask of all valid flags */ + #ifdef CONFIG_SPU_FS_MODULE int register_spu_syscalls(struct spufs_calls *calls); -- cgit v1.1 From 867672777964b9309e4e914fe097648c938b67b2 Mon Sep 17 00:00:00 2001 From: Arnd Bergmann Date: Wed, 4 Oct 2006 17:26:21 +0200 Subject: [POWERPC] spufs: add infrastructure for finding elf objects This adds an 'object-id' file that the spe library can use to store a pointer to its ELF object. This was originally meant for use by oprofile, but is now also used by the GNU debugger, if available. In order for oprofile to find the location in an spu-elf binary where an event counter triggered, we need a way to identify the binary in the first place. Unfortunately, that binary itself can be embedded in a powerpc ELF binary. Since we can assume it is mapped into the effective address space of the running process, have that one write the pointer value into a new spufs file. When a context switch occurs, pass the user value to the profiler so that can look at the mapped file (with some care). Signed-off-by: Arnd Bergmann Signed-off-by: Paul Mackerras --- include/asm-powerpc/spu.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) (limited to 'include') diff --git a/include/asm-powerpc/spu.h b/include/asm-powerpc/spu.h index 83b6dae..e73ea00 100644 --- a/include/asm-powerpc/spu.h +++ b/include/asm-powerpc/spu.h @@ -201,6 +201,24 @@ static inline void unregister_spu_syscalls(struct spufs_calls *calls) /* + * Notifier blocks: + * + * oprofile can get notified when a context switch is performed + * on an spe. The notifer function that gets called is passed + * a pointer to the SPU structure as well as the object-id that + * identifies the binary running on that SPU now. + * + * For a context save, the object-id that is passed is zero, + * identifying that the kernel will run from that moment on. + * + * For a context restore, the object-id is the value written + * to object-id spufs file from user space and the notifer + * function can assume that spu->ctx is valid. + */ +int spu_switch_event_register(struct notifier_block * n); +int spu_switch_event_unregister(struct notifier_block * n); + +/* * This defines the Local Store, Problem Area and Privlege Area of an SPU. */ -- cgit v1.1