dynamically loading data?

i have a function that loads datasets, which is quite complex, but you can use it as such:

Code:

---


loadset setname ("filename.h5");

---

How can I make a wrapper function that I can use to dynamically load these sets?
For example, It is up to the user to determine how many sets must be loaded simultaneously. So I would need to dynamically comeup with 'setname', such as, if the user enters "5", then this:

Code:

---

for (int i = 0; i < chunk; i++){
loadset string("setname" + i) ("filename1.h5");

---

where string("setname" + i) is the variable's name. obviously this code won't work.

perhaps putting it in a struct or class, and then dynamically creating a class or structure, how would I do such a thing?

Quote:

---

Originally Posted by elninio

Code:

---

loadset setname ("filename.h5");

---

---

Is this a declaration or code?

Are you trying to create the filename, as in filename.h1, filename.h2, filename.h3? If so, use a stringstream to create the filename.

Otherwise, how are the sets broken up? All in the same file?

loadset is a class. setname is an object of loadset. it loads a dataset from the file "filename.h5" into memory. i'm trying to load filenames dynamically into memory depending on how many a user specifies. each object of loadset should correspond to a unique file.

So, setname1 corresponds to file1.h5 , setname2 corresponds to file2.h5 etc..

I'm thinking of using a recursive linked list. But the problem will be specifying loading a different filename for each node

Ok, use a stringstream to create the filename.

Use an array (or probably a vector) to store each loadset. So instead of setname1, it would be loadset_list[0].

Depending on how expensive it is to copy your loadset object, you might consider using a ptr_vector from boost or a vector of shared_ptrs (or a carefully managed vector of raw pointers).

Quote:

---

Originally Posted by Daved

Ok, use a stringstream to create the filename.

Use an array (or probably a vector) to store each loadset. So instead of setname1, it would be loadset_list[0].

Depending on how expensive it is to copy your loadset object, you might consider using a ptr_vector from boost or a vector of shared_ptrs (or a carefully managed vector of raw pointers).

---

I didn't know that was possible - I have no idea how to implement this, I don't even have a guess!

these files are datasets which contain subdirectories and matrices, do you mean to store a pointer to each loadset? I cringe just writing this:

int array[10];

array[1]fail

ok i give up.

I have no idea how it is possible to store an entire dataset in an array, and I have no idea how to declare a pointer to it (doubting if it is even possible). Sorry :/

Quote:

---

Originally Posted by Daved

Ok, use a stringstream to create the filename.

Use an array (or probably a vector) to store each loadset. So instead of setname1, it would be loadset_list[0].

Depending on how expensive it is to copy your loadset object, you might consider using a ptr_vector from boost or a vector of shared_ptrs (or a carefully managed vector of raw pointers).

---

loadset has to be declared dynamically, thus it needs to have an indentifyer that is also declared dynamically.

Nope, it doesn't work:

Code:

---


'array' has a previous declaration as 'int array [50]'

---

for

Code:

---

int array [50];
dataset_loader array[0]("pic3.h5");
dataset_loader array[0]("pic2.h5");

---

can someone show me how i could make a datastructure that would dynamically load these files

struct node(
dataset_loader set("file1.h5");
node *next;
)

I could make a loop that generates through this, but how would I make it so that "file1.h5" would be changed to file2 file3 file4 etc.. for each iteration of the loop?

Don't use your own linked list, use a vector or array. You have the right idea, except your array should be an array of dataset_loader objects, not an array of int.

Is there a way to create a dataset_loader object without specifying a filename? If you create an array, you cannot specify the filename in the constructor because you won't know what the filenames are until the program is running.

You should really use vector for this. Are you familiar with vectors? Now would be a good time to do a little research on them because they will help you do exactly what you want.

Is there a way to create a dataset_loader object without specifying a filename?

no. there isn't you have to create objects of that class by specifying a filename. the reason for that is because you would the run the following function

dataset_buf filename = load_data_set("/root/folder/folder2");

Basically it returns a reference to a dataset buffer. The you can start accessing its members such as filename(x,y,z);

You keep changing the names you use and it is confusing me. What do the following mean:

loadset
dataset_loader
dataset_buf
load_data_set

?


Can you copy the loadset?

Code:

---

loadset setname ("file1.h5");
loadset copy = setname;

---

sorry,

these are my commands:

dataset_loader someset("filename");
dataset_buf somebuf = someset.load_data_set("/folder/folder2");
somebuf(x,y,z);

dataset_loader and dataset_buf are functions in the dataset_loader class

Can you copy the loadset?
no you can't do this. datasets are .h5 files, these are complex hierarchies made of folders and matrices.

I mean can you make a copy of the class like in my example (except change loadset to dataset_loader). Will that compile?

If not, then you need to have an array (or vector) of pointers, then use new to allocate your dataset_loader objects and save the pointer in the array (or vector). Even if you can copy the object this might be a good choice for efficiency reasons. However it is more error prone because you have to manage the memory.