Segfaults while using Greengrass IPC Publisher CPP Example

Question

Looking for advice/guidelines on how to use the Greengrass IPC publish sample code in a multi-threaded C++ application that will have many asynchronous calls making Publish requests.

I have read the following guide, (https://docs.aws.amazon.com/greengrass/v2/developerguide/interprocess-communication.html), in particular the "Best practices for subscription handlers" section. Is there an equivalent section for publishing IPC messages ?

Our current approach:
Run the following code once, in the main/init function:

ApiHandle apiHandle(g_allocator);
Io::EventLoopGroup eventLoopGroup(1);
Io::DefaultHostResolver socketResolver(eventLoopGroup, 64, 30);
Io::ClientBootstrap bootstrap(eventLoopGroup, socketResolver);
GreengrassCoreIpcClient ipcClient(bootstrap);

And then, when we wish to publish an IPC message, we call the following function:

int publish_ipc_message(GreengrassCoreIpcClient * ipcClient, String topic, String message, int timeout) {
IpcClientLifecycleHandler ipcLifecycleHandler;

auto connectionStatus = ipcClient->Connect(ipcLifecycleHandler).get();  
// Check for pre-existing connection status  
if(connectionStatus.StatusToString() == "EVENT_STREAM_RPC_CONNECTION_ALREADY_ESTABLISHED") {  
    std::cout << "Event Stream Already Established" << std::endl;  
} else if (!connectionStatus) {  
    std::cerr << "Failed to establish IPC connection: " << connectionStatus.StatusToString() << std::endl;  
    return 1;  
}  
  
PublishToTopicRequest request;  
Vector<uint8_t> messageData({message.begin(), message.end()});  
BinaryMessage binaryMessage;  
binaryMessage.SetMessage(messageData);  
PublishMessage publishMessage;  
publishMessage.SetBinaryMessage(binaryMessage);  
request.SetTopic(topic);  
request.SetPublishMessage(publishMessage);  

PublishToTopicOperation operation = ipcClient->NewPublishToTopic();  
auto activate = operation.Activate(request, nullptr);  
activate.wait();  

auto responseFuture = operation.GetResult();  
if (responseFuture.wait_for(std::chrono::seconds(timeout)) == std::future_status::timeout) {  
    std::cerr << "Operation timed out while waiting for response from Greengrass Core." << std::endl;  
    exit(-1);  
}  

auto response = responseFuture.get();  
if (response) {  
    std::cout << "Successfully published to topic: " << topic << std::endl;  
} else {  
    // An error occurred.  
    std::cout << "Failed to publish to topic: " << topic << std::endl;  
    auto errorType = response.GetResultType();  
    if (errorType == OPERATION_ERROR) {  
        auto *error = response.GetOperationError();  
        std::cout << "Operation error: " << error->GetMessage().value() << std::endl;  
    } else {  
        std::cout << "RPC error: " << response.GetRpcError() << std::endl;  
    }  
     exit(-1);  
}  

return 0;

We occasionally receive a segfault from this publish_ipc_message() function when it is called, my guess is that passing the same ipcClient to each call of the function is responsible for the segfault.

Any guidance on how to approach multi-threaded async use of IPC publish would be appreciated.

Answer

Hi GreengrassUser,  
  
Thanks for reporting this, looking at the code you posted, I cannot see a specific piece that might be causing this. One note is that you do not need to attempt Connect() every time you want to publish the message, you can do that only once when you create the IPC client in your component initialization/start up code and keep using the IPC client for any IPC operations, any number of times in the component process later, it also means that using it from several threads to publish many messages as you're doing should work fine. The SDK maintainers would be able to help better in identifying/fixing this issue, could you please create an issue in the SDK github repo to get their help ? https://github.com/aws/aws-iot-device-sdk-cpp-v2/issues   
  
Thanks,  
Shagupta

Answer

Issue solved by update to the sdk on github

Answer

I have created an issue in the Github, I was able to produce segfaults with the sample app for publishing IPC messages. I am still awaiting AWS follow-up on that.   
  
See: https://github.com/aws/aws-iot-device-sdk-cpp-v2/issues/327

Answer

I have attached a snippet of a gdb backtrace from when the segfault occurs:

2021-09-28T15:17:10.959Z  Thread 304 "enc_que0:src" received signal SIGSEGV, Segmentation fault.. 
2021-09-28T15:17:10.960Z  [Switching to Thread 0x7eaa1fa7c0 (LWP 760)]. 
2021-09-28T15:17:10.960Z  0x0000005561527ac8 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x7ee8006640) at /usr/include/c++/7/bits/shared_ptr_base.h:154. 
2021-09-28T15:17:10.960Z  154		    _M_dispose();. 
2021-09-28T15:17:11.263Z [#0  0x0000005561527ac8 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (this=0x7ee8006640) at /usr/include/c++/7/bits/shared_ptr_base.h:154. 
2021-09-28T15:17:11.264Z   #1  0x0000005561527ac8 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (this=0x7eaa1f9ad0, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/shared_ptr_base.h:684. 
2021-09-28T15:17:11.264Z  #2  0x0000005561527ac8 in std::__shared_ptr<std::__future_base::_State_baseV2, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (this=0x7eaa1f9ac8, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/shared_ptr_base.h:1123. 
2021-09-28T15:17:11.264Z  #3  0x0000005561527ac8 in std::shared_ptr<std::__future_base::_State_baseV2>::~shared_ptr() (this=0x7eaa1f9ac8, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/shared_ptr.h:93. 
2021-09-28T15:17:11.264Z   #4  0x0000005561527ac8 in std::__basic_future<Aws::Eventstreamrpc::RpcError>::~__basic_future() (this=0x7eaa1f9ac8, __in_chrg=<optimized out>) at /usr/include/c++/7/future:671. 
2021-09-28T15:17:11.264Z  #5  0x0000005561527ac8 in std::future<Aws::Eventstreamrpc::RpcError>::~future() (this=0x7eaa1f9ac8, __in_chrg=<optimized out>) at /usr/include/c++/7/future:761. 
2021-09-28T15:17:11.264Z  #6  0x0000005561527ac8 in publish_ipc_message(Aws::Greengrass::GreengrassCoreIpcClient*, std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> >, int) (ipcClient=<optimized out>, topic="<snipped for readability>"..., message=..., timeout=<optimized out>) at /root/source/deepstream-app/src/greengrass_ipc_pub.cpp:56.

_____________________________________________________________________________________________-

Edited by: GreengrassUser on Sep 28, 2021 9:20 AM

Segfaults while using Greengrass IPC Publisher CPP Example

Relevant content