Mod Multi Packet

From EMule Wiki
(Difference between revisions)
Jump to: navigation, search
m (SCT)
(Undo revision 3006 by Izegiwera (Talk))
 
(26 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
= Mod Multi Packet Specifications =
 
= Mod Multi Packet Specifications =
The mod multi packet is designed as a default container for file Reask related data’s, like ICS or SCT. It can be sent over TCP as well as over UDP. It is in the new mod prot optionality but it is strongly recommended to use it to save overhead and have a larger opcode pool available.
+
The mod multi packet is designed as a default container for file Reask related data’s, like ICS or SCT. It can be sent over TCP as well as over UDP. It is in the new mod prot optional but it is strongly recommended to use it to save overhead and have a larger opcode pool available.
  
 
The TCP multi Packet(Answer) have the following format:
 
The TCP multi Packet(Answer) have the following format:
Line 15: Line 15:
 
The TCP packet is sent imminently after the normal eMule Multi Packet, the answer is sent when the mod multi packet is received.
 
The TCP packet is sent imminently after the normal eMule Multi Packet, the answer is sent when the mod multi packet is received.
  
The UDP packet is similar but it does not have the file hash on the top, because it is not sent together with the file reask ping bus as an answer on the "File Reask Ping Acknowledge".
+
The UDP packet have the exact same format, but it isn't sent together with the file reask ping but as an answer on the "File Reask Ping Acknowledge".
  
 
The Extended 4 way file reask ping looks like following:
 
The Extended 4 way file reask ping looks like following:
Line 28: Line 28:
 
This way was chosen because it allows sending the Mod Multi Packet also to Low ID clients without using the Kad Buddy.
 
This way was chosen because it allows sending the Mod Multi Packet also to Low ID clients without using the Kad Buddy.
  
The UDP multi Packet(Answer) have the following format:
+
Note: Support for the "Extended 4 way file reask ping" have to be announced separately in the Mod info tag FILE_STATUS_EXTENSIONS
<pre>
+
Entries till data end:
+
[uint8 1]                  // Sub Opcode
+
[uint16 2] len            // entry length
+
[data]                    // content
+
Next Entry:
+
...
+
</pre>
+
  
 
= Mod Multi Packet Applications =
 
= Mod Multi Packet Applications =
  
== ICS v1/v2 ==
+
== ICS ==
ICS Extends the regular chunk status with the status of incomplete chunks, it sends a separated status bit field listing chunks that are incomplete but not empty. This is use full for [[Intelligent Chunk Selection]] as well as for release related features (v2).
+
ICS (Version 2) Extends the regular chunk status with the status of incomplete chunks, it sends a separated status bit field listing chunks that are incomplete but not empty. This is use full for [[Intelligent_chunk_selection]] as well as for release related features.
  
 
The ICS informations should be accepted in every Multi Packet because all 4 variants should use the same parsing function.
 
The ICS informations should be accepted in every Multi Packet because all 4 variants should use the same parsing function.
Line 47: Line 39:
 
The mod multi packet entry for ICS and other Part based status informations shell have the following format:
 
The mod multi packet entry for ICS and other Part based status informations shell have the following format:
 
<pre>
 
<pre>
[uint16 2] count          //PartCount
+
[uint16 2] count          // PartCount
 
[bitField count/8]        // bit encoded status informations
 
[bitField count/8]        // bit encoded status informations
 
</pre>
 
</pre>
  
=== Protocol v1 ===
+
The ICS version used for mod Prot is v2 and it sends the Status in request answers, as well as in the request itself.
The v1 version sends the informations only in file request answers over TCP, it don't sent them over UDP and it also don't sends them to the source on a file request.  
+
  
=== Protocol v2 (Neo) ===
+
The purpose of sending the data in file request is to allow the releasing clients have a better overview witch parts are needed as well as don't apply part hiding for a particular client on parts that he have incomplete.
The Neo v2 extension enables sending of the status informations over UDP and in file request. The purpose of sending the data in file request is to allow the releasing clients have a better overview witch parts are needed as well as don't apply part hiding for a particular client on parts that he have incomplete.
+
As well as heaving the status for a4af sources as well in order to can better select the most needed chinks.
 +
 
 +
Note: the old v1 sends the informations only in file request answers over TCP, it don't sends them to the source in a file request.
  
 
== SCT ==
 
== SCT ==
Line 62: Line 55:
 
The mod multi packet entry for SCT have the following format:
 
The mod multi packet entry for SCT have the following format:
 
<pre>
 
<pre>
[uint16 2] count       // Map Count
+
[uint8 1] opts  // [bits 1-3] // block divider how many blocks are represented by one bit (only 1, 2, 3, 4, 7 are valid)
 +
                // [bits 4-8]  // reserved
 +
[uint8 1] count         // Map Count
 +
if count == 0xff [uint16 2] long count  // just in case
 
// block maps for selected parts
 
// block maps for selected parts
 
[uint16 2] part // part number
 
[uint16 2] part // part number
[bitField56 7]          // [bits 1-53] block status
+
[bitField 7/Div]       // what blocks are done
                        // [bits 54-55] reserved
+
                        // [bits 56] determines if the map contains verified blocks
+
 
// next map
 
// next map
 
</pre>
 
</pre>
 +
 +
Please note that no more than 15 Maps should be sent at once due to the overhead, a recommended limit is min(15,max(5,PartCount/10)).
 +
The Maps are sent progressive, only when all have been sent they are resent from the begin and only on TCP, on UDP no resent is done after all maps are out.
 +
 +
== RPS ==
 +
Real Part Status is an additional extension of the part status informations, it contains the informations witch parts have been hidden/blocked by releasing client. The Mon Multi packet entry have the exact same format as ICS. It can be sent under 2 different opcodes indicating if the parts are only hidden and are downloadable or if they are completely blocked.
  
 
= Appendix  =
 
= Appendix  =
 +
== UDP Handling ==
 +
Whether the Extended mod file status informations are to be sent as well over UDP or not is determined by a separated flag (UDP Mod File Status). The flag tels us if the extended status informations enabled for TCP by mod flags are to be sent as well over UDP.
 +
 +
== Status Handling ==
 +
 +
The client must be able to handle the situation when the remote client don't send the extended status informations on  every reask(answer) (it must not drop the old status until new status is received).
 +
 +
Clients should only send the extended status over UDP when something in the status changed since the last time.
 +
 +
The extended status should Always be sent over TCP because we don't know if the remote client restarted in the mean time, and lost the old status, if he hadn't he should used a UDP reask.
  
 
== OpCodes ==
 
== OpCodes ==
 +
{| style="font-size: 85%; text-align: Left;Border="1";
 +
! rowspan=2 | NAME
 +
! rowspan=2 | VALUE
 +
|-
 +
|
 +
|-
 +
|OP_MODMULTIPACKET
 +
| '0x11'
 +
|-
 +
|OP_MODMULTIPACKETANSWER
 +
| '0x12'
 +
|-
 +
|
 +
|-
 +
|SOP_FILEINCSTATUS
 +
| '0x8E'
 +
|-
 +
|SOP_SUBCHUNKMAPS
 +
| '0x53'
 +
|-
 +
|}
 +
  
OP_MODMULTIPACKET 0x11
+
[[category:Mod_protocol_extensions]]
OP_MODMULTIPACKETANSWER 0x12
+
ET_INCOMPLETEPARTS 0x3D
+

Latest revision as of 09:06, 24 November 2010

Contents

Mod Multi Packet Specifications

The mod multi packet is designed as a default container for file Reask related data’s, like ICS or SCT. It can be sent over TCP as well as over UDP. It is in the new mod prot optional but it is strongly recommended to use it to save overhead and have a larger opcode pool available.

The TCP multi Packet(Answer) have the following format:

[hash128 16]               // FileHash
Entries till data end:
[uint8 1]                  // Sub Opcode
[uint16 2] len             // entry length
[data]                     // content
Next Entry:
...

The TCP packet is sent imminently after the normal eMule Multi Packet, the answer is sent when the mod multi packet is received.

The UDP packet have the exact same format, but it isn't sent together with the file reask ping but as an answer on the "File Reask Ping Acknowledge".

The Extended 4 way file reask ping looks like following:

A                 B
FR_PING    --->                   // this step may be done over a Kad buddy !!!
          <---    FR_PING_ACK
MOD_MP     --->
          <---    MOD_MP_ANS

This way was chosen because it allows sending the Mod Multi Packet also to Low ID clients without using the Kad Buddy.

Note: Support for the "Extended 4 way file reask ping" have to be announced separately in the Mod info tag FILE_STATUS_EXTENSIONS

Mod Multi Packet Applications

ICS

ICS (Version 2) Extends the regular chunk status with the status of incomplete chunks, it sends a separated status bit field listing chunks that are incomplete but not empty. This is use full for Intelligent_chunk_selection as well as for release related features.

The ICS informations should be accepted in every Multi Packet because all 4 variants should use the same parsing function.

The mod multi packet entry for ICS and other Part based status informations shell have the following format:

[uint16 2] count          // PartCount
[bitField count/8]        // bit encoded status informations

The ICS version used for mod Prot is v2 and it sends the Status in request answers, as well as in the request itself.

The purpose of sending the data in file request is to allow the releasing clients have a better overview witch parts are needed as well as don't apply part hiding for a particular client on parts that he have incomplete. As well as heaving the status for a4af sources as well in order to can better select the most needed chinks.

Note: the old v1 sends the informations only in file request answers over TCP, it don't sends them to the source in a file request.

SCT

The Sub Chunk Transfer feature is mainly only a block status exchange it is sent in file reask answers associated, mod multi packets, not in file requests associated ones, but it can be handled in booth.

The mod multi packet entry for SCT have the following format:

[uint8 1] opts  // [bits 1-3] // block divider how many blocks are represented by one bit (only 1, 2, 3, 4, 7 are valid)
                // [bits 4-8]  // reserved
[uint8 1] count         // Map Count
if count == 0xff [uint16 2] long count  // just in case
// block maps for selected parts
[uint16 2] part		// part number
[bitField 7/Div]        // what blocks are done
// next map

Please note that no more than 15 Maps should be sent at once due to the overhead, a recommended limit is min(15,max(5,PartCount/10)). The Maps are sent progressive, only when all have been sent they are resent from the begin and only on TCP, on UDP no resent is done after all maps are out.

RPS

Real Part Status is an additional extension of the part status informations, it contains the informations witch parts have been hidden/blocked by releasing client. The Mon Multi packet entry have the exact same format as ICS. It can be sent under 2 different opcodes indicating if the parts are only hidden and are downloadable or if they are completely blocked.

Appendix

UDP Handling

Whether the Extended mod file status informations are to be sent as well over UDP or not is determined by a separated flag (UDP Mod File Status). The flag tels us if the extended status informations enabled for TCP by mod flags are to be sent as well over UDP.

Status Handling

The client must be able to handle the situation when the remote client don't send the extended status informations on every reask(answer) (it must not drop the old status until new status is received).

Clients should only send the extended status over UDP when something in the status changed since the last time.

The extended status should Always be sent over TCP because we don't know if the remote client restarted in the mean time, and lost the old status, if he hadn't he should used a UDP reask.

OpCodes

NAME VALUE
OP_MODMULTIPACKET '0x11'
OP_MODMULTIPACKETANSWER '0x12'
SOP_FILEINCSTATUS '0x8E'
SOP_SUBCHUNKMAPS '0x53'
Personal tools