Hi Stan,
thanks. I already got the adapters. In case I encounter a problem, I will post a specific questions.
Best regards,
Christoph
Von: Smith, Stan [mailto:***@intel.com]
Gesendet: Montag, 19. August 2013 18:12
An: Fab Tillier; Christoph MÃŒller
Cc: '***@lists.openfabrics.org'
Betreff: RE: Use of I/O completion port with Winverbs
Christoph,
Located in the winOFED src tree under ulp\dapl2\dapl\NetworkDirect\NdHelper.cpp, specifically NdOpenAdapter().
Also in dapl_nd_device.c : dapls_ib_open_hca() could provide some NDv2 device handling insights.
W.r.t. to which ND device, NdQueryAddressList() will return a list of ND devices on the system; also see dapli_GetIPaddr().
For the DAPL case, I match a device name from dat.conf ( hca0 ) and use the device name trailing instance number as an index into the NdQueryAddressList() of ND devices.
Stan.
From: ofw-***@lists.openfabrics.org<mailto:ofw-***@lists.openfabrics.org> [mailto:ofw-***@lists.openfabrics.org] On Behalf Of Fab Tillier
Sent: Sunday, August 18, 2013 12:19 PM
To: Christoph MÃŒller
Cc: '***@lists.openfabrics.org'
Subject: Re: [ofw] Use of I/O completion port with Winverbs
Ah, I didnât realize Mellanox included the headers in their redist, ok.
The docs for NDv2 are in the pipeline, but thereâs quite a backlog for our tech writers so I donât have an ETA unfortunately. The combination of NDv1 and NDK docs should get you on the right path in the meantime, and you can always send me questions.
Provider discovery is largely the same as NDv1, though you call DllGetClassObject for the IND2Provider interface directly, rather than going through the IClassFactory interface.
You should be able to interop between ND and librdma code, as well as between Windows and Linux no problem. If it doesnât work then thereâs a bug in the drivers somewhere. Of course youâll be limited by the capabilities of NDv2 (no remote atomic, only reliable connected, no immediate data, etc.)
-Fab
From: Christoph MÃŒller [mailto:***@visus.uni-stuttgart.de]
Sent: Sunday, August 18, 2013 12:01 PM
To: Fab Tillier
Cc: '***@lists.openfabrics.org'
Subject: AW: Use of I/O completion port with Winverbs
Hi Fab,
thanks. I already got access to the IDN2Provider on Friday - the headers are also included in Mellanox' distribution just as Tzachi said. The problem on my side was that I was looking for some import libraries. I finally found some NDv1 documentation on MSDN explaining how to get the root object via the Winsock provider catalogue. However, there does not seem to be any NDv2 documentation yet, but the differences seem to be subtle.
As I have only limited time, I will probably not be able to rewrite all of my code at once. I think that my NDv2 code will be able to talk to librdma peers as long as I use only basic functionality like remote read/write or inline send/receive. Is that correct?
Best regards,
Christoph
Von meinem Windows Phone gesendet
________________________________
Von: Fab Tillier<mailto:***@microsoft.com>
Gesendet: â18.â08.â2013 07:38
An: Christoph MÃŒller<mailto:***@visus.uni-stuttgart.de>
Cc: '***@lists.openfabrics.org'<mailto:***@lists.openfabrics.org>
Betreff: RE: Use of I/O completion port with Winverbs
Hi Christoph,
You can get the ND header files (included in the NetworkDirect DDK zip file) here:
http://www.microsoft.com/en-us/download/details.aspx?id=36043
There is no dependency on the HPC Pack - you just need drivers that support ND (ideally NDv2, which the Mellanox 4.40 release does).
Using ND should give you the greatest hardware independence too, though by design ND exposes the lowest common denominator functionality of iWARP and InfiniBand, so some functionality is not available (e.g. RDMA atomics).
Donât hesitate to ask if you have questions.
-Fab
From: ofw-***@lists.openfabrics.org<mailto:ofw-***@lists.openfabrics.org> [mailto:ofw-***@lists.openfabrics.org] On Behalf Of Christoph MÃŒller
Sent: Thursday, August 15, 2013 6:04 AM
To: Tzachi Dar; ***@lists.openfabrics.org<mailto:***@lists.openfabrics.org>
Subject: Re: [ofw] Use of I/O completion port with Winverbs
Thank you. I was not aware that the SDK including NDv2 is distributed with Mellanoxâ drivers. In this case, I will give it a try.
Thanks again,
Christoph
Von: Tzachi Dar [mailto:***@mellanox.com]
Gesendet: Donnerstag, 15. August 2013 14:43
An: Christoph MÃŒller; ***@lists.openfabrics.org<mailto:***@lists.openfabrics.org>
Betreff: RE: Use of I/O completion port with Winverbs
You only need to download Mellanox drivers 4.40 for windows, and you will have nd support.
http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_sw_drivers
Thanks
Tzachi
From: Christoph MÃŒller [mailto:***@visus.uni-stuttgart.de]
Sent: Thursday, August 15, 2013 3:32 PM
To: Tzachi Dar; ***@lists.openfabrics.org<mailto:***@lists.openfabrics.org>
Subject: AW: Use of I/O completion port with Winverbs
ndinstall is part of the HPC pack distribution? My main problem with ND is to find out how to get it...
Best regards,
Christoph
Von meinem Windows Phone gesendet
________________________________
Von: Tzachi Dar<mailto:***@mellanox.com>
Gesendet: â15.â08.â2013 14:28
An: Christoph MÃŒller<mailto:***@visus.uni-stuttgart.de>; ***@lists.openfabrics.org<mailto:***@lists.openfabrics.org>
Betreff: RE: Use of I/O completion port with Winverbs
Please note that the windows APIs are changing from time to time and ND is probably the one that will stay longer since it is supported by MS.
I donât think that you need to install the hpc pack for it. You simply need to run ndinstall and then you can start running nd programs.
Thanks
Tzachi
From: Christoph MÃŒller [mailto:***@visus.uni-stuttgart.de]
Sent: Thursday, August 15, 2013 3:21 PM
To: Tzachi Dar; ***@lists.openfabrics.org<mailto:***@lists.openfabrics.org>
Subject: AW: Use of I/O completion port with Winverbs
Actually, I am in the process of deciding which API to use (although I have existing librdma code). NetworkDirect indeed looks promising, but as I understand, I need HPC Pack 2012 for that which I cannot install on our cluster in the short term.
Best regards,
Christoph
Von meinem Windows Phone gesendet
________________________________
Von: Tzachi Dar<mailto:***@mellanox.com>
Gesendet: â15.â08.â2013 14:15
An: Christoph MÃŒller<mailto:***@visus.uni-stuttgart.de>; ***@lists.openfabrics.org<mailto:***@lists.openfabrics.org>
Betreff: RE: Use of I/O completion port with Winverbs
can you write your application with Network Direct?
If so, there are overlapped objects there that you should be able to connect to the completion ports.
Thanks
Tzachi
From: ofw-***@lists.openfabrics.org<mailto:ofw-***@lists.openfabrics.org> [mailto:ofw-***@lists.openfabrics.org] On Behalf Of Christoph MÃŒller
Sent: Thursday, August 15, 2013 3:10 PM
To: ***@lists.openfabrics.org<mailto:***@lists.openfabrics.org>
Subject: [ofw] Use of I/O completion port with Winverbs
Hi all,
I was wondering whether it is possible to use CreateIoCompletionPort etc. with IWVConnectEndpoints. I can see that the implementation inherits a handle from CWVBase, but I do not see how I could use this information for creating an IOCP and, of course, I do not know whether it would work. Furthermore, I would prefer continuing to use the RDMA library, but I see absolutely no way to create an IOCP here. So me questions are:
Is it possible to use IOCPs with winverbs?
Is it possible to use IOCPs with the RDMA library?
Is there sample code for either of it?
Thanks in advance,
Christoph
--
Visualisierungsinstitut der UniversitÀt Stuttgart
Allmandring 19, 70569 Stuttgart
E-Mail: ***@visus.uni-stuttgart.de<mailto:***@visus.uni-stuttgart.de>
Telefon: (07 11) 685 - 88626