passing an array pointer between two processes?

[celu]

Hi guys, I have this maya plugin which creates an array on heap.Now I want to read this array from my directX application.Basically for passing some real time data between the plugin and my DirectX application.My plugin writes to the array and my Directx application only reads those values.How do I acheive this? Im basically new to Inter Process Communication.Any suggestions as to what I need to look into would be really helpful.Im using win32 and VS 2005 IDE..if that helps...Thx.

[Kenneth Gorking]

Either memory mapping (http://msdn2.microsoft.com/en-us/library/aa914748.aspx) or named pipes (http://en.wikipedia.org/wiki/Named_pipe#Named_pipes_in_Windows) should do the trick.

[dave_]

There are loads (http://en.wikipedia.org/wiki/Inter-process_communication#Implementations) of ways of doing it


Personally I like sockets because it can then be extended across networks too.
But it sounds like you want shared memory (http://en.wikipedia.org/wiki/Shared_memory). Named pipes seems like a good approach to this.

[celu]

thx a lot..ill look into them..will get back if i need further clarifications...thx :)

[lstockman]

There are loads (http://en.wikipedia.org/wiki/Inter-process_communication#Implementations) of ways of doing it


Personally I like sockets because it can then be extended across networks too.
But it sounds like you want shared memory (http://en.wikipedia.org/wiki/Shared_memory). Named pipes seems like a good approach to this.

Just by chance, how would you pass an array using sockets? I'm not that great at network programming and this is one of my challenges that I look forward to tackeling so that I can increase my skill. Any help would be appreciated.

[Reedbeta]

If it's a TCP socket, you just write the array data to the socket in order, and read it back on the other end. Of course if the data contains pointers you have to change them to indices or else they won't make sense (since they won't have the same addresses on the destination side).

[celu]

but passing a pointer to an array (allocated on the heap) between two processes on the same machine wont be a problem rite? unlike in a network...the pointer will still point to the same array on the receiving process also init?

[Nils Pipenbrinck]

On Win32 there is an api for that. Lookup GlobalAlloc. It will allocate memory and give you a handle. You can lock and unlock the handle to get a pointer to work with.

The idea behind this is that it's valid to pass the handle between processes. The kernel then maps the memory into the process address space. All processes that work with that handle will see the same physical memory.

[Reedbeta]

but passing a pointer to an array (allocated on the heap) between two processes on the same machine wont be a problem rite? unlike in a network...the pointer will still point to the same array on the receiving process also init?

No, because each process executes in its own virtual address space. Processes don't normally have access to each others' memory. However you can use GlobalAlloc as Nils suggested, or you can also use the memory-mapped I/O API, which does more or less the same thing.

[celu]

I allocated a memory block using the GlobalAlloc() function and obtained a handle to that block.I filled the block with my array data by locking the handle.Then I passed the memory block handle to my second process using a named pipe.Now I have the memory handle in my second process,It still points to the same address as in the first process(I verified this using the debugger).But when i try to lock the handle it returns a null pointer.Im not able get a pointer to my array data that I loaded in the first process.What could be going wrong? I cant seem to find the reason...my initial hunch was that it had something to do with the allocation flag of my GlobalAlloc() function i tried changing the allocation flag from GMEM_FIXED to GMEM_MOVEABLE.But that made no difference.

[Reedbeta]

If you already have a named pipe, why not just send the data through that?

[Nils Pipenbrinck]

Celu,

You almost got it. Global Memory handles are serialized. Only one process can lock it. So after process A has written some stuff it has to unlock the handle. Then process B can lock it and read/write the data.

You might see the same pointer in all processes. If so the kernel has allocated shared address-space for you (ain't that nice?). You can't however access the data without locking. Just locking once and remembering the pointer won't work.

If you have a pipe and you can send data, use it. I personally find global memory handles easier to deal with (I pass them around using window-messages or across dll boundaries). It's just a mater of preference though.

[.oisyn]

When you use shared memory (keyword: memory mapped files), both processes can read and write to that memory simultaneously. Also, you can make the OS to map that memory onto a specific address within your address space, and if both processes map using the same address, you can even use regular pointers (as long as they point to data within the shared memory region of course)

[celu]

I have a DirectX application(process A) that needs to use an array created by maya plugin(process B).Process B updates this array continuously,and Process A needs to read this updated information from this array during each frame.I wanted the most efficient way of getting this array data from Process B to Process A. Process B only writes to the array and Process A only reads from the array.I thought of creating a global memory block in process B,copy the array on to this block.Then pass the handle to this global memory block to process A(using named pipes).Once i receive the handle in process A.I could then lock it to read the array data from the global memeory block into Process A.This way Process A could read this memory block when ever it needs to by locking & unlocking it and process B could write to it in the same way.I think this would be better than using named pipe for passing the entire array between the two processes during each frame.Moreover passing the array between the processes at frame rate speeds using named pipes would introduce synchronizing delays that might affect the frame rate init? By passing the memory handle I need to use the named pipes only at the start(to pass the memory handle).After that, the two process will have their own copy of the global memory block handle.And process B can write to that memory block and process A can read from it.At frame rate speeds this would be better init?

[Reedbeta]

Well, global memory blocks can only be locked by one process, so you'd need some kind of synchronization mechanism between the two to know when you can pass control. Maybe you should use memory-mapped I/O instead; see CreateFileMapping (http://msdn2.microsoft.com/en-us/library/aa366537.aspx) (passing INVALID_HANDLE_VALUE for the first argument lets you create a named mapping object that can be shared, rather like a named pipe). You should be able to map a view of the file in both processes and use this as shared memory without the need to lock/unlock (though you'll have to keep a synchronization bit in the data, or use a mutex (http://msdn2.microsoft.com/en-us/library/ms684266.aspx) or similiar to prevent race conditions).

[.oisyn]

Also, a shared memory handle and mutices can be created and opened using global names. That way, you don't have to pass handles to the other application using named pipes or whatever (which are tedious for non-global handles like a mutices, as you first have to duplicate the handle for the other process)

[celu]

ya i think memory-mapped I/O will fit my needs correctly...think ill have to remove the pipes and redo my app with memory mapped I/O..will keep you updated on the progress and thx a lot all you guys :)

[saqibbs]

My problem is very similar to what the original poster intended to do in the first place.

I have two processes A and B, A passes some data to B through named-pipe, (in reality it is passing a pointer to some data in A's memory space) so when B receives the data, it gets the correct value of the pointer, but the pointer which B received doesn't point to a memory location in A's memory space. (I know this because the data accessed by B's pointer received from A is not the same value assigned by A before passing the pointer)

Here are two points.

i) My understanding is that process A and B don't share the same memory space hence passing pointers to each other is fruitless so I need to set up shared memory between A&B (I need confirmation on this point)

ii) But looking at a driver code, this is exactly what is being done, when I need to read from a device, I pass a pointer to the location where I want the data from the device and the driver returns when it has written data in that location. The location where I want my data to be stored is of course in my memory space but the driver is stable able to access that space.
(so my understanding is that in (ii) it happens because the driver is under the kernel-control so kernel allows such things but two processes in user-space aren't allowed to do such things)

I have modified a small client-server model taken from ,
http://developers.sun.com/solaris/articles/named_pipes.html#2a
which proves one, but I need confirmation that what I have done is correct and what I have concluded is correct too.

Thanks

Server CODE:

#include <stdio.h>
#include <errno.h>
#include <ctype.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include "full_duplex.h" /* For name of the named-pipe */

int main(int argc, char *argv[])
{
int rdfd, wrfd, ret_val, count, numread;
char buf[MAX_BUF_SIZE];

unsigned long *buff;
int *buffPtr;
buff = (unsigned long *)buf;

ret_val = mkfifo(NP1, 0666);

if ((ret_val == -1) && (errno != EEXIST)) {
perror("Error creating the named pipe");
exit (1);
}

/* Open the first named pipe for reading */
rdfd = open(NP1, O_RDONLY);

/* Read from the first pipe */
numread = read(rdfd, buf, MAX_BUF_SIZE);
printf("Num Read %d %ld\n", numread, buff[0]);

buffPtr = (int *)buff[0];

printf("%ld\n", buffPtr);
printf("%d\n", *buffPtr);

}

Client CODE:

#include <stdio.h>
#include <errno.h>
#include <ctype.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include "full_duplex.h" /* For name of the named-pipe */

int main(int argc, char *argv[])
{
int wrfd, rdfd, numread, sample;
int *address;
char rdbuf[MAX_BUF_SIZE];
unsigned long *buff;

address = &sample;
sample = 33;
buff = (unsigned long *)rdbuf;
buff[0] = (unsigned long)&sample;

printf("%d %ld %ld\n", sample, &sample, buff[0]);


/* Open the first named pipe for writing */
wrfd = open(NP1, O_WRONLY);

/* Write to the pipe */
write(wrfd, rdbuf, 8);

exit(0);

}

[Reedbeta]

Okay, first of all, please use the code tags when you post code. I have added them for you this time.

Now as for your question.
(i) Yes.
(ii) Yes.

[saqibbs]

Thanks for clearing that up. Here are a few more questions.

So in my client code I created any anonymous memory segment which was supposed to be shared, (i used mmap to create it) and then I passed to pointer to this memory the server, and the pointer value on the server side is the same as the client yet the data doesn't match.

i) In Stevens book (Unix Networking, IPC) the memory is shared across child and parent

ii) How can we share memory between two unrelated processes.

iii) Should I use System V IPC (shmget, shmid etc) compared to using BSD sharing (mmap)??

Thanks

[Reedbeta]

Well, I've never used *nix memory sharing facilities, so I could be wrong. But in Windows, what you have to do is create a named shared memory block in one process, pass the name to the other process, and the other process asks for that block to be mapped. Creating a shared memory block in one process doesn't automatically map the same address range in all child processes; each process that wants access to the shared block has to ask for it individually (and could have it mapped to a different address range in their own address space). The processes sharing a block don't have to be in a parent/child relationship (in fact that's irrelevant to the mechanics of sharing memory).