Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[21612] Data sharing performance improvement #5209

Open
wants to merge 35 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
00df4b9
Hotfix: Secure simple participants with `initialpeers` over `TCP` mat…
Mario-DL Aug 27, 2024
e02a52d
Fix the problem that gcc-9 cannot compile
paxifaer Aug 28, 2024
9c8bb04
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.cpp
paxifaer Sep 2, 2024
eeb1381
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.cpp
paxifaer Sep 2, 2024
d12a810
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.cpp
paxifaer Sep 2, 2024
d2eacdb
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
b446cd4
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.cpp
paxifaer Sep 2, 2024
242456a
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
3f179ac
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
291fa89
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
3dfdb20
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
75b7480
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.cpp
paxifaer Sep 2, 2024
549d97a
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.cpp
paxifaer Sep 2, 2024
91ed3a6
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
1e6d9aa
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.cpp
paxifaer Sep 2, 2024
2ec01b6
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.cpp
paxifaer Sep 2, 2024
ca455e2
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
f9153ef
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
8122e05
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
64e4230
Update src/cpp/fastdds/xtypes/dynamic_types/AnnotationDescriptorImpl.hpp
paxifaer Sep 2, 2024
7a19061
Revert missing noexcept suggested by JesusPoderoso.
paxifaer Sep 2, 2024
30328eb
Set string arguments as const references (#5179)
JesusPoderoso Sep 2, 2024
28a3371
Improve resilience against clock adjustments (#5018)
ma30002000 Sep 2, 2024
2b80fa7
TypeLookupService: also register the minimal created from the receive…
richiware Sep 5, 2024
8063e60
Use eProsima-CI action to install Qt (#5186)
JesusPoderoso Sep 6, 2024
a2098d6
Compared with the traditional separate locking method, Acquire-Releas…
paxifaer Sep 7, 2024
b62bc1b
Merge branch 'master' into data_sharing
paxifaer Sep 19, 2024
6de8155
Merge branch 'eProsima:master' into data_sharing
paxifaer Sep 19, 2024
34ba4c6
Compared with the traditional separate locking method, Acquire-Releas…
paxifaer Sep 7, 2024
9eaae61
Fix uncrustify
paxifaer Oct 3, 2024
860d8dd
Merge branch 'data_sharing' of https://github.com/paxifaer/Fast-DDS i…
paxifaer Oct 3, 2024
1e95b84
fix uncrustify 2
paxifaer Oct 3, 2024
0bc9d2e
fix duplicate code
paxifaer Oct 3, 2024
4ce305e
For thread safety, add locks. Mark the usage restrictions of zero cop…
paxifaer Oct 3, 2024
fa752fa
Mark the usage restrictions of zero copy in test.
paxifaer Oct 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class AnnotationDescriptorImpl : public virtual AnnotationDescriptor

virtual ~AnnotationDescriptorImpl() = default;

traits<DynamicType>::ref_type type() const noexcept override
traits<DynamicType>::ref_type type() const override
{
return type_;
}
Expand Down
13 changes: 8 additions & 5 deletions src/cpp/rtps/DataSharing/DataSharingListener.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ void DataSharingListener::stop()
listening_thread_.join();
}

void DataSharingListener::process_new_data ()
void DataSharingListener::process_new_data()
{
EPROSIMA_LOG_INFO(RTPS_READER, "Received new data notification");

Expand All @@ -138,7 +138,10 @@ void DataSharingListener::process_new_data ()
// It is safe to 'forget' any change now
notification_->notification_->new_data.store(false);
// All places where this is set to true is locked by the same mutex, memory_order_relaxed is enough
writer_pools_changed_.store(false, std::memory_order_relaxed);
// Using Acquire-Release semantics can avoid some blocking problems in traditional locking, such as deadlock and priority inversion.
// This usually means higher concurrency performance because threads don't have to wait for locks to be released.
// Through Acquire-Release, you can achieve finer-grained control over specific variables or data structures without having to lock the entire resource or object, which can reduce contention and improve concurrency.
writer_pools_changed_.store(false, std::memory_order_release);

// Loop on the writers looking for data not read yet
for (auto it = writer_pools_.begin(); it != writer_pools_.end(); ++it)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit worried about the validity of this iterator changing during the next few lines (until we get to std::shared_ptr<ReaderPool> pool = it->pool;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there was a problem with effectiveness here, it would have been exposed before. I'll keep an eye on this later, and if there's a problem I'll fix it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The places where writer_pools_ is modified are protected by taking mutex_.
Your PR is removing the guard on the mutex_ during the traversal of writer_pools_ here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The places where writer_pools_ is modified are protected by taking mutex_. Your PR is removing the guard on the mutex_ during the traversal of writer_pools_ here.

Don't worry about this, I use the lock_free mechanism here. It is thread safe.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writer_pools_ is a std::vector<WriterInfo> which is not thread-safe. If DataSharingListener::remove_datasharing_writer() is called on one thread while another thread is inside DataSharingListener::process_new_data(), an invalidated iterator on the vector could be dereferenced

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right. But I tested it many times under multi-thread conditions and there was no problem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-locked

Expand Down Expand Up @@ -199,7 +202,7 @@ void DataSharingListener::process_new_data ()
}
}

if (writer_pools_changed_.load(std::memory_order_relaxed))
if (writer_pools_changed_.load(std::memory_order_acquire))
{
// Break the while on the current writer (it may have been removed)
break;
Expand All @@ -209,7 +212,7 @@ void DataSharingListener::process_new_data ()
// Lock again for the next loop
lock.lock();

if (writer_pools_changed_.load(std::memory_order_relaxed))
if (writer_pools_changed_.load(std::memory_order_acquire))
{
// Break the loop over the writers (itearators may have been invalidated)
break;
Expand Down Expand Up @@ -313,4 +316,4 @@ std::shared_ptr<ReaderPool> DataSharingListener::get_pool_for_writer(

} // namespace rtps
} // namespace fastdds
} // namespace eprosima
} // namespace eprosima
4 changes: 4 additions & 0 deletions test/performance/latency/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,10 @@ The utility offers several options:
| --security=[true/false] | Enable/disable DDS security |
| --certs=\<directory> | Directory with the certificates. Used when security is enable |

**Container Tips**
Shared_memory and data_sharing technologies have pre-allocated memory pools underneath. This memory value occupies the system's free memory segment. To put it simply, the size of
the sent data packet will affect the amount of allocated memory, which is probably shm: 1:5 data_sharing 1:31. If the data packet size is 100MB data, in data_sharing mode
At least 3.1g of memory is required. docker run -it --shm-size=3.1g my-image.

**Publication options**

Expand Down
10 changes: 10 additions & 0 deletions test/performance/throughput/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ The utility offers several options:
| --certs=\<directory> | Directory with the certificates. Used when security is enable |



**Publication options**

| Option | Description |
Expand Down Expand Up @@ -141,6 +142,10 @@ The file with the demands has the format of each line the data size separated of
This examples will execute four tests: one sending bursts of 100 samples of 16 bytes, other test sending bursts of 1000 samples of 16 bytes,
, other sending bursts of 100 samples of 32 bytes and the last one sending bursts of 1000 samples of 32 bytes.

**Container Tips**
Shared_memory and data_sharing technologies have pre-allocated memory pools underneath. This memory value occupies the system's free memory segment. To put it simply, the size of
the sent data packet will affect the amount of allocated memory, which is probably shm: 1:5 data_sharing 1:31. If the data packet size is 100MB data, in data_sharing mode
At least 3.1g of memory is required. docker run -it --shm-size=3.1g my-image.

### Examples

Expand Down Expand Up @@ -185,3 +190,8 @@ The python scripts offers several options:
| -t \<seconds> | Test time in seconds. Default is *1 second* |
| -r \<file> | A CSV file with recovery time |
| -f \<file> | A file containing the demands |

**Container Tips**
Shared_memory and data_sharing technologies have pre-allocated memory pools underneath. This memory value occupies the system's free memory segment. To put it simply, the size of
the sent data packet will affect the amount of allocated memory, which is probably shm: 1:5 data_sharing 1:31. If the data packet size is 100MB data, in data_sharing mode
At least 3.1g of memory is required. docker run -it --shm-size=3.1g my-image.