« Analyzing crash dump … | Home | MBS Plugin Stand bei … »

Reading crash reports on MacOS

After the Windows crash report from earlier today where we couldn't do much, here is an example for one where we could help on MacOS.

When you find a crash report in /Library/Logs/DiagnosticReports folder (for server software) or ~/Library/Logs/DiagnosticReports (for user software), you can try to read where it crashed and maybe avoid that code or fix the data processed by the code to avoid the problem.

The crash report we check today start with this basic info:

Process: fmscwpc [19163]
Path: /Volumes/*/fmscwpc
Identifier: fmscwpc
Version: 0
Code Type: X86-64 (Native)
Parent Process: fmserver_helperd [19089]
Responsible: fmscwpc [19163]
User ID: 503

Date/Time: 2019-09-26 13:32:54.009 +0200
OS Version: Mac OS X 10.13.6 (17G8030)
Report Version: 12
Anonymous UUID: 3BB48127-8D6B-D79A-BCF9-09FEA3400430

As you can read, we have the process name fmscwpc, which is FileMaker Server Custom Web Publishing Client as far as I know. This is the process of FileMaker Server handling requests for WebDirect. It runs here on MacOS 10.13 and you see the timestamp when it happens. When you can reproduce the issue, you may get a few crash reports, so having times may help to distinguish them. Next section shows details on the problem:

Time Awake Since Boot: 10000 seconds

System Integrity Protection: enabled

Crashed Thread: 103

Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000000

Termination Signal: Segmentation fault: 11
Termination Reason: Namespace SIGNAL, Code 0xb
Terminating Process: exc handler [0]

VM Regions Near 0:
__TEXT 000000010a8f5000-000000010aa92000 [ 1652K] r-x/rwx SM=COW /Users/USER/*

We read here that crash 103 crashed with a bad memory access error. Some memory was accessed, e.g. a buffer overrun where you only have 100 elements, but you try to access the 200th element and nobody checked bounds.

Scrolling down through over 100 threads, we find the thread 103 and here is the call stack:

Thread 103 Crashed:
0 fmscwpc 0x000000010a9a841e iwp::IWPLayoutParts::GetDataForSpecifiedPopover(int, unsigned long long, std::__1::map, std::__1::allocator > >&, std::__1::map, std::__1::allocator > >&, bool, short, int, int) + 726
1 fmscwpc 0x000000010a9ae642 iwp::IWPLayoutAccessor::GetPopoverData(int, unsigned long long, int, short, int, bool, std::__1::map, std::__1::allocator > >&, std::__1::map, std::__1::allocator > >&) const + 64
2 fmscwpc 0x000000010a976ae5 iwp::IWPServiceImpl::getDataForCurrentPopover(layout::LayoutDataResult&, context::Context const&, int, int, int) + 217
3 fmscwpc 0x000000010a951835 iwp::IWPServiceProcessor::process_getDataForCurrentPopover(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) + 493
4 fmscwpc 0x000000010a94be6a iwp::IWPServiceProcessor::process_fn(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__1::basic_string, std::__1::allocator >&, int, void*) + 106
5 fmscwpc 0x000000010a94bc3a iwp::IWPServiceProcessor::process(boost::shared_ptr, boost::shared_ptr, void*) + 114
6 libthriftnb-0.9.2.dylib 0x000000010acafbc3 apache::thrift::server::TNonblockingServer::TConnection::Task::run() + 243
7 libthrift-0.9.2.dylib 0x000000010abd34ce apache::thrift::concurrency::ThreadManager::Task::run() + 30
8 libthrift-0.9.2.dylib 0x000000010abd2d55 apache::thrift::concurrency::ThreadManager::Worker::run() + 597
9 libthrift-0.9.2.dylib 0x000000010ac10872 apache::thrift::concurrency::PthreadThread::threadMain(void*) + 178
10 libsystem_pthread.dylib 0x00007fff66025661 _pthread_body + 340
11 libsystem_pthread.dylib 0x00007fff6602550d _pthread_start + 377
12 libsystem_pthread.dylib 0x00007fff66024bf9 thread_start + 13

You read it from bottom up, so the thread starts and runs thread manager functions in apache framework to do tasks. One of the tasks is in IWPServiceProcessor class to process some requests. It calls the getDataForCurrentPopover function here to get data and the inner call to GetDataForSpecifiedPopover causes the problem by access some data, which is not there. So this problem seems to be related to loading a popover.

For a crash report always check further down the versions of the loaded libraries. When I see Adobe PDF plugins, I recommend to delete them and if I spot old plugins, I also recommend to update those. The last section is the memory summary. If the application uses a lot of memory, it may have been crashed by running out of available memory. But our example crash report just shows 1.2 GB, which seems to be a reasonable size for this process.

After talking back to client, they could see the crash happens when the layout with the popover is used. When the popover was removed and later replaced with a new one, the problem went away. The client is happy and I hope he also reported it to Claris Inc., so it can be fixed in a future version of FileMaker Server, if it is not fixed already and this server is not yet updated.
15 10 19 - 09:20