Using StarPU automatic memory allocation with partitioning
Outline
I am trying to use StarPU's automatic memory allocation by setting home_node
to -1
when registering a new handle (for matrix data).
In addition, I also need to dynamically generate partitions of this handle.
When clearing the partition, I run into the following problem:
[starpu][_starpu_data_unpartition_submit][assert failure] gathering node different from home node is currently not supported
_starpu_data_unpartition_submit: Assertion `gather_node == initial_handle->home_node || gather_node == -1' failed.
This is generated from around this line, where it also says #warning FIXME: better choose gathering node
.
How to reproduce
Here is a minimum working example for reproducing the issue. Set AUTOALLOC
to true
or false
to toggle between allocation modes. The code only works (for me) when I set AUTOALLOC
to false
.
#include "starpu.h"
#define AUTOALLOC true
struct starpu_data_filter f_vert = {
.filter_func = starpu_vector_filter_block, .nchildren = 4
};
void init_cpu(void* buffers[], void*) {
double* matrix = (double*)STARPU_VECTOR_GET_PTR(buffers[0]);
uint nx = STARPU_VECTOR_GET_NX(buffers[0]);
for (uint i=0; i<nx; ++i) matrix[i] = 0;
}
int main(int argc, char** argv) {
STARPU_CHECK_RETURN_VALUE(starpu_init(NULL), "init");
struct starpu_codelet init_cl;
starpu_codelet_init(&init_cl);
init_cl.cpu_funcs[0] = init_cpu;
init_cl.nbuffers = 1;
init_cl.modes[0] = STARPU_W;
starpu_data_handle_t parent;
#if AUTOALLOC
starpu_vector_data_register(&parent, -1, 0, 16, sizeof(double));
#else
double data[16];
starpu_vector_data_register(
&parent, STARPU_MAIN_RAM, (uintptr_t)data, 16, sizeof(double)
);
#endif
starpu_data_handle_t children[f_vert.nchildren];
starpu_data_partition_plan(parent, &f_vert, children);
starpu_task_insert(&init_cl, STARPU_W, children[0], 0);
starpu_data_partition_clean(parent, f_vert.nchildren, children);
starpu_data_unregister_submit(parent);
starpu_shutdown();
}
I am still new to StarPU, so there is a good chance I am making a simple mistake here, but I cannot figure out what that might be - this along with the warning for developers in the code where the assertion fails motivated me to create this issue.