Dynamically instantiating datatypes using templates

[Shree]

I'm aware that you can use templates to handle multiple datatypes.

However, when instantiating a templated object you still must specify the type. E.g.

DECLARATION:


template<typename T>
struct point {
T x T y;};


INSTANTIATION:

point<int> p; // implicit instance of point

I know you can use the C++ typeid operator to extract a string containing the type name of an object. E.g


Obj X;

cout << "object X is of type" << typeid(X).name();


Is it possible to use this operator to dynamically instantiate an object of the appropriate type? i.e.

point<typeid(X).name()> p; // instantiate instance of point with appropriate data type

If not, is there another way to achieve this assuming we are only talking about basic types (not user defined types).

Thus far the only way I have seen this done is via a series of if/else typeid comparisons for each type you want to handle:


ObjX;
int i, char c, float f;

if ( typeid (i)==typied (X))
point<int> p;
else
if ( typeid (c)==typied (X))
point<char> p;
else
if ( typeid (f)==typied (X))
point<float> p;


I was hoping to avoid having to maintain a hardcoded list of such comparisons if possible.

Thanks

[.oisyn]

ObjX;
int i, char c, float f;

if ( typeid (i)==typied (X))
point<int> p;
else
if ( typeid (c)==typied (X))
point<char> p;
else
if ( typeid (f)==typied (X))
point<float> p;

First of all, this won't work as each 'p' is defined in the scope of each if function, and will not be visible otherwise. But in this particular example, you know the type of X (it's Obj), and you usually always know the type since you need to use it at compile-time anyway. So can you provide us with the exact problem where you want the solution for? It's a good chance there's another and perhaps better way to handle things.

[hovermonkey]

No, because in C++ the compiler does full type-checking and generates code for each type you want to instantiate the template with.
If you were to try to do that at runtime you would need to have the compiler available to generate code at runtime, and that would be very bad, especially on embedded systems.

[.oisyn]

He's not saying "at runtime", he wants to use the type of a variable to instantiate a template with. Some compilers support the typeof operator (as does C++0x), which makes it possible (point<typeof(X)> p), but with the current C++ standard it isn't.

[Shree]

Thanks for the replies. oisyn basically hit the nail on the head.

I'm modifying some code which reads in data from a file and builds a data structure with it.
However, it currently only supports 8 bit scalars (i.e unsigned chars) but I need it to also handle 32 bit scalars (i.e floats).

The author of the code suggested the best way to do this was using templates as this would allow it to support multiple data types going forwards.

From your comments, the typeof operator can do this but it's not part of the C++ standard so I'd rather not use it

My if/else code example was poorly written. Sorry.

I've seen code where people use typeid comparisons to identify the type of an object passed into a function and was wondering if this is a good way to solve my problem.

[hovermonkey]

Oh in that case you could probably do something like this:

template < class X > point<X>* create_point( const X& val )
{
return new point<X>();
}
...
point<int>* pIntPoint = create_point( 3 );
point<float>* pFloatPoint = create_point( 1.234f );
point<char*>* pStringPoint = create_point( "hello" );

however you can't have a series of if/else comparisons like you wrote as you are redefining p each time, and with different types. You'd have to give your point<X> classes a common non-templated base class, and cast the returned pointer into a pointer to the base class. Example:

class GeneralPoint
{
public:
virtual void Print() = 0;
};

template < class X > class Point : public GeneralPoint
{
X a, b;
public:
void Print() { cout << a << "," << b; }
};


Now you can do

int i; float f;
GeneralPoint* pPoint;
if (some_condition) { pPoint = create_point( i ); }
else { pPoint = create_point( f ); }
pPoint->Print();

[.oisyn]

I'm modifying some code which reads in data from a file and builds a data structure with it.
However, it currently only supports 8 bit scalars (i.e unsigned chars) but I need it to also handle 32 bit scalars (i.e floats).

The author of the code suggested the best way to do this was using templates as this would allow it to support multiple data types going forwards.

From your comments, the typeof operator can do this but it's not part of the C++ standard so I'd rather not use it
Well, in your example you needed the typeof, but you can usually figure out the type in another way - I mean, at some point in your program you know the type. So why are you having problems using that type as a template argument? Can't you just use a typedef for the type X, so you can use that typedef to instantiate your templates as well? If not, why not? Please give an exact use-case of the problem you're trying to solve (instead of a solution you want it to solve it with).

[Shree]

Well, in your example you needed the typeof, but you can usually figure out the type in another way - I mean, at some point in your program you know the type. So why are you having problems using that type as a template argument?


The program currently assumes that the data in the file being read is composed of 8-bit scalars. It therefore does not correctly process 32-bit scalar data. I'm trying to modify it to handle both so I must determine the type of the data when the file is read and use this to create a data structure of the appropriate type.


Can't you just use a typedef for the type X, so you can use that typedef to instantiate your templates as well? If not, why not?


Sorry - I don't follow you. The type X of the data is unknown until the file is read (see above).


Please give an exact use-case of the problem you're trying to solve (instead of a solution you want it to solve it with).

Below is an extract from the code showing one of the variables in the data structure and it's get/set functions which assume the data is composed of 8-bit scalars.

I need to modify all such variables and functions to also handle 32-bit scalars.



typedef unsigned char ST;

class OctreeVolume
{
....
public:
ST isoval;

inline void set_isovalue(double isovalue)
vals->isoval = (ST)isovalue;

inline int get_isovalue() const
return vals->isoval;
....
}

Is this what you are after?

[J22]

So if you want to make OctreeVolume which is composed of different type instead, you just remove the ST typedef and make OctreeVolume a template class instead:

template<typename ST>
class OctreeVolume
{
...
};

[Shree]

So if you want to make OctreeVolume which is composed of different type instead, you just remove the ST typedef and make OctreeVolume a template class instead:

template<typename ST>
class OctreeVolume
{
...
};

This would work if ALL the data in the class was of type ST but unfortunately it isn't . I only posted an extract as the entire class is too long to list here. Also, structs are used to create the octree structure which lie outside the class scope so I will have to modify them separately.

It appears that the solution isn't a simple one and I've been advised to consider implementing an abstract factory class which will recquire an overhaul of the current design. I was hoping to implement a quick solution to get things working first, similar to that provided by hovermonkey above.

I apologise if I've not explained the problem well enough. If you need further clarification I'ill do my best to provide it.

[J22]

Well, there are lots of things which aren't clear from your problem description thus people are not really able to help you but only make random guesses of the problem. If you could write little SimplifiedOctreeVolume which demonstrates the problem, that would help, or just toss the entire OctreeVolume class definition and implementation somewhere. Guessing game just drives people mad ;)

[.oisyn]

Well, you do know what you are going to read, right? I mean, what determines whether a particular piece of data is char or int? In what level is this truely dynamic? Isn't the fileformat fixed?

[Shree]

Well, you do know what you are going to read, right? I mean, what determines whether a particular piece of data is char or int? In what level is this truely dynamic? Isn't the fileformat fixed?

The idea is for the code to support different data types transparently by determining if the points are unsigned chars or floats (or whatever else is supported) and creating an appropriate data structure to hold them.

So you supply it with a data file and it either creates an octree or exits with an error explaining that the format you are providing is unsupported (and lists the supported formats FYI).

Currently I must manually convert the data (using the "unu" utility or something similar) from float to unsigned char before the program can use it.

[kusma]

Shree: perhaps you want to make the entire loading-code templated?
something like:
template <typename T>
DataContainter<T> loadFile(const char *filename)
{
DataContainter<T> container;
FILE *fp = fopen(filename, "rb");
for (int i = 0; i < 100; ++i)
{
T t;
fread(&t, sizeof(T), 1, fp);
container.push_back(t);
}
return container;
}

DataContainter<float> floatData = loadFile("floatdata.bin");
DataContainter<signed char> charData = loadFile("char.bin");

[Nautilus]

Currently I must manually convert the data (using the "unu" utility or something similar) from float to unsigned char before the program can use it.
I take that the data to read is not in binary form.
Would you post 2 short samples of this data file?
1 with BYTE data
and 1 with, say, FLOAT data.

Ciao ciao : )

[J22]

So all the data in one octree is either char or float, but you can't have both at the same time in an octree? There are couple of ways you can go then:
1) you can have octree as a template class and instantiate octree of a specific type. if you try to load octree of different type, do conversion to the type (and possibly print a warning) or exit with an error.
2) you can have octree base class and derive templated octree class from that. at loading stage determine the type and instantiate matching octree type and use the octree through the base class interface.
So, it basicly depends if you want to be able to have client code that transparently is able to deal with different types of octrees, or if you only want to be able to use specific type of octree in specific situation in your code.

[Shree]

Firstly, thanks a lot for all your replies.

I take that the data to read is not in binary form.

The data is in binary form. The files contain either 8-bit unsigned chars or 32-bit floats. I'd like the code to be able to read in a file, detemine which data type it contains and create an octree accordingly.


So, it basicly depends if you want to be able to have client code that transparently is able to deal with different types of octrees, or if you only want to be able to use specific type of octree in specific situation in your code.

I've already converted the octree into a template class but it seems I must implement polymorphism to achieve the former:


2) you can have octree base class and derive templated octree class from that. at loading stage determine the type and instantiate matching octree type and use the octree through the base class interface.

However, this brings me back to the problem of how to instantiate an octree of the appropriate type when I load the data:


at loading stage determine the type and instantiate matching octree type

I thought of using some form of RTTI (e.g the typeid operator?) as described in my OP.

[Shree]

. You'd have to give your point<X> classes a common non-templated base class, and cast the returned pointer into a pointer to the base class. Example:

class GeneralPoint
{
public:
virtual void Print() = 0;
};

template < class X > class Point : public GeneralPoint
{
X a, b;
public:
void Print() { cout << a << "," << b; }
};





One question I have about this example is what happens to functions which have template types as arguments?

For example, if we add to the above class a function which takes an argument of type X:


template < class X> class Point : public GeneralPoint
{
X a;
public:
void Print() { cout << a; }

void confine_a(X max_a, X min_a)
{
a = MAX(min_a, a);
a = MIN(max_a, a);
}

};


How would I add a virtual declaration of this function to the abstract base class?

[Nautilus]

The data is in binary form. The files contain either 8-bit unsigned chars or 32-bit floats. I'd like the code to be able to read in a file, detemine which data type it contains and create an octree accordingly.
Look, if it's binary there *must* be a way to tell beforehand how to read the thing. There's no escape.
Even a plain TXT is binary. But you find alphanumeric characters in it, and Notepad is instructed to interpret 0D 0A sequences as carriage return + line feed.
It's a format.
Windows Media Player could read your data file and try to carve an MPG movie out of it. But something will tell WMP that it's not an MPG.
It's the file format.

Get what I mean?
If your program can't do the same, the file format is incomplete (for the *new* use you make of it) and has to be modified.

Ciao ciao : )

[J22]

However, this brings me back to the problem of how to instantiate an octree of the appropriate type when I load the data
Oh, you just store the information to the file. Let say you store one byte to the beginning of the file, which is 0 if it's char type octree and 1 if it's float type octree:

BaseOctreeVolume *LoadOctree(ifstream &s)
{
BaseOctreeVolume *o=0;
char type;
s>>type;
switch(type)
{
case 0: o=new OctreeVolume<char>(); break;
case 1: o=new OctreeVolume<float>(); break;
default: Error("unknown octree format");
}
o->Load(s);
return o;
}

[Shree]

Oh, you just store the information to the file. Let say you store one byte to the beginning of the file, which is 0 if it's char type octree and 1 if it's float type octree:


This is viable if modifying the file prior to reading it is an option. Unfortunately, I'm working towards eventually receiving this data over a network in real-time so any such manipulation/conversion must be performed at the receiving end.

In other words, the octree generation code needs to 'transparently' handle the different data formats.

The approach I'm currently considering is:

(1) Guess the data type from the file extension (and exit with an error if it's not supplied/recognised)

(2) Change the octree generation code to generate 32 bit octrees instead and then add some type conversion code before this which converts 8 bit data into 32 bit data. This removes the need to make the octree generation generic (using templates and factory classes) but I will still need to re-write it to support 32 bit scalars which won't be easy.

[J22]

Doesn't matter if you receive the data over the network or from a file. There is no manipulation of the file but it's just a file format used when the octree is stored into a file. Determining the type from the first byte is no different from guessing the type from file extension, except that the actual data format is more robust solution because you can't break it by renaming the file. It sounds to me that you want to be able to read files which are stored assuming the octree is always char type, but the format (if made with any sense in mind at all) has some kind of versioning which you can bump up and add the option for different types.

Anyway, this is just basic data serialization stuff you are talking about and has nothing to do with templates or what so ever.

[Shree]

I think something along the lines of what you suggested above (storing the datatype and reading it in with the file) will work and isa good solution to my problem so I am now proceeding with this approach.

Thanks a lot for your help with this!

[Shree]

Give your point<X> classes a common non-templated base class, and cast the returned pointer into a pointer to the base class. Example:

class GeneralPoint
public:
virtual void Print() = 0;
};

template < class X > class Point : public GeneralPoint
{
X a, b;
public:
void Print() { cout << a << "," << b; }
};




When using this approach how much of the derived class definition must be declared in the abstract base class definition?

All the member functions (i.e. the class interface) need declaring as pure virtual functions - pure to ensure the class is abstract, virtual so that they can be overriden in the derived class.

But do I also need to declare in the base class all the member variables (i.e. the class data) in the derived class?

I'm concerned that if I don't do this then the compiler won't allocate any memory for these variables so that when I cast from a derived to a base class pointer I will lose data......

OR

Am I just complicating the issue and is it infact possible to assign directly to a pointer to the template class without explicitly providing a type at compile-time ? e.g.


int i; float f;
Point<T>* p;
if (some_condition)
{ p = create_point( i ); }
else
{ p = create_point( f ); }

[.oisyn]

I'm concerned that if I don't do this then the compiler won't allocate any memory for these variables so that when I cast from a derived to a base class pointer I will lose data......

Yes, you will lose access to the data, because the base doesn't hold the data. But you don't lose the data itself, because it's a pointer. Now if it weren't a pointer, you will get object slicing, because you're creating a copy of only the base class, and not the derived part (which makes sense as there isn't any storage allocated to hold the derived data). But a pointer just points to the base part of the object, and you can get the whole object back by doing another cast (to it's derived class).


Am I just complicating the issue and is it infact possible to assign directly to a pointer to the template class without explicitly providing a type at compile-time ?
No, you can't. Templates and their instantiations are a compile-time thing, you can't resolve these issues at runtime.

[Shree]

oisyn - thanks for your replies to my query which also reminded me about object slicing and explained why it doesn't happen with pointers. However, I still I think need to refresh my understanding through some reading as my knowledge is obviously still patchy!

I used the method outlined by hovermonkey in my code at it works great.

Thankyou all for your responses which are helping me to progress :)