Realm.NET SDK 5.01 throws exception when filter returns null

[VagueGit]

On Realm.NET SDK 5.01
Windows 10.0.19041
Visual Studio 2019 16.7.3
Realm Cloud v3.28.2

Running VS in Debug mode Realm.NET SDK Version 4.3.0 this snippet returns null when none found and allows a null check

string filter = $"OrderDetail.OrderDetailId == '{rOrderDetailWithoutJob.OrderDetailId}'";
Data.Job rJob = realm.All<Data.Job>().Filter(filter).FirstOrDefault();
if (rJob != null)
...

On Realm.NET SDK Version 5.1.0 the line
Data.Job rJob = realm.All<Data.Job>().Filter(filter).FirstOrDefault();
throws an exception when rJob is null. The exception message is

Key not found

OrderDetail.OrderDetailId is the primary key and it is not null
rJob is null so of course the primary key is null

Have I missed a breaking change?

Here is the stack trace

   at Realms.NativeException.ThrowIfNecessary(Func`2 overrider)
   at Realms.CollectionHandleBase.TryGetObjectAtIndex(Int32 index, ObjectHandle& objectHandle)
   at Realms.RealmResultsVisitor.VisitMethodCall(MethodCallExpression node)
   at Realms.RealmResultsProvider.Execute(Expression expression)
   at Realms.RealmResultsProvider.Execute[T](Expression expression)
   at Wibble.Services.DataService.<>c__DisplayClass33_0.<RestoreLinksFromOrderDetailsToJobs>b__0() in C:\Repos\FSL\Wibble\Services\DataService.cs:line 397
   at Realms.Realm.Write(Action action)
   at Wibble.Services.DataService.RestoreLinksFromOrderDetailsToJobs() in C:\Repos\FSL\Wibble\Services\DataService.cs:line 391
   at Wibble.Services.DataService.CheckOrphans() in C:\Repos\FSL\Wibble\Services\DataService.cs:line 361
   at Wibble.Services.DataService.<CheckDbAsync>d__25.MoveNext() in C:\Repos\FSL\Wibble\Services\DataService.cs:line 264
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at UUWP.Device.StartDevice.<StartAsync>d__0.MoveNext() in C:\Repos\FSL\UUWP\Device\StartDevice.cs:line 17
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at UUWP.App.<OnLaunched>d__5.MoveNext() in C:\Repos\FSL\UUWP\App.xaml.cs:line 41

[nirinchev]

Seems more like a bug - I'll try to reproduce it and get back to you. If you have a small isolated project that exhibits this behavior, it'll speed things up.

[VagueGit]

If you have a small isolated project that exhibits this behavior, it'll speed things up.

Sorry @nirinchev but the snippet is from a large project that jumps through lots of hoops to access the dataset, so not something I can easily tease out into a sample repo.

[nirinchev]

What's the schema of the involved classes? I can't seem to be able to reproduce with a simple unit test. Also, what's the value for filter when the exception is thrown?

[VagueGit]

Thank you for giving this your attention @nirinchev

rOrderDetailWithoutJob is of type OrderDetail

using Realms;
using System;
using System.Reflection;

namespace Wibble.Data
{
    internal class OrderDetail : RealmObject
    {
        [PrimaryKey]
        [MapTo(nameof(OrderDetailId))]
        private string _OrderDetailId { get; set; }
        public string OrderDetailId
        {
            get => _OrderDetailId;
            set
            {
                if (string.IsNullOrWhiteSpace(value))
                    throw new ArgumentNullException($"{MethodBase.GetCurrentMethod().Name} cannot be blank for {this.GetType().Name}");
                _OrderDetailId = value;
            }
        }
        [MapTo(nameof(Order))]
        private Order _Order { get; set; }
        public Order Order
        {
            get => _Order;
            set
            {
                if (value == null)
                    throw new ArgumentNullException($"{MethodBase.GetCurrentMethod().Name} cannot be blank for {this.GetType().Name}");
                _Order = value;
            }
        }
        public Job Job { get; set; }

        [Required]
        [MapTo(nameof(ItemCode))]
        private string _ItemCode { get; set; }
        public string ItemCode
        {
            get { return _ItemCode; }
            set
            {
                if (string.IsNullOrWhiteSpace(value))
                    throw new ArgumentNullException($"{MethodBase.GetCurrentMethod().Name} cannot be blank for {this.GetType().Name}");
                _ItemCode = value;
            }
        }
        [Required]
        [MapTo(nameof(ItemDescription))]
        private string _ItemDescription { get; set; }
        public string ItemDescription
        {
            get { return _ItemDescription; }
            set
            {
                if (string.IsNullOrWhiteSpace(value))
                    throw new ArgumentNullException($"{MethodBase.GetCurrentMethod().Name} cannot be blank for {this.GetType().Name}");
                _ItemDescription = value;
            }
        }

        [MapTo(nameof(UnitCost))]
        private double _UnitCost { get; set; }
        public decimal UnitCost
        {
            get => (decimal)Math.Round(_UnitCost, 2);
            set => _UnitCost = Math.Round((double)value, 2);
        }
        public int NrUnits { get; set; }

        [MapTo(nameof(UnitPrice))]
        private double _UnitPrice { get; set; }
        public decimal UnitPrice
        {
            get => (decimal)Math.Round(_UnitPrice, 2);
            set => _UnitPrice = Math.Round((double)value, 2);
        }

        private OrderDetail()
        {
        }
        public OrderDetail(Order order, string orderDetailId, string itemCode, string itemDescription)
        {
            this.Order = order;
            this.OrderDetailId = orderDetailId;
            this.ItemCode = itemCode;
            this.ItemDescription = itemDescription;
        }
    }
}

rJob is of type Job

using Realms;
using System;
using System.Linq;
using System.Reflection;
using Wibble.Services;

namespace Wibble.Data
{
    internal class Job : RealmObject
    {
        [PrimaryKey]
        [MapTo(nameof(JobId))]
        private string _JobId { get; set; }
        public string JobId
        {
            get => _JobId;
            set
            {
                if (string.IsNullOrWhiteSpace(value))
                    throw new ArgumentNullException($"{MethodBase.GetCurrentMethod().Name} cannot be blank for {this.GetType().Name}");
                _JobId = value;
            }
        }
        public DateTimeOffset? DateCompleted { get; set; }
        public DateTimeOffset? DateCollected { get; set; }
        public DateTimeOffset? DateReminder { get; set; }
        [MapTo(nameof(OrderDetail))]
        private OrderDetail _OrderDetail { get; set; }
        public OrderDetail OrderDetail
        {
            get => _OrderDetail;
            set
            {
                if (value == null)
                    throw new ArgumentNullException($"{MethodBase.GetCurrentMethod().Name} cannot be blank for {this.GetType().Name}");
                _OrderDetail = value;
            }
        }
        [MapTo(nameof(JobType))]
        private JobType _JobType { get; set; } = DataService.DefaultJobType();
        public JobType JobType
        {
            get => _JobType;
            set
            {
                if (value == null)
                    throw new ArgumentNullException($"{MethodBase.GetCurrentMethod().Name} cannot be blank for {this.GetType().Name}");
                _JobType = value;
            }
        }
        [MapTo(nameof(Location))]
        private Location _Location { get; set; } = DataService.DefaultLocation();
        public Location Location
        {
            get => _Location;
            set
            {
                if (value == null)
                    throw new ArgumentNullException($"{MethodBase.GetCurrentMethod().Name} cannot be blank for {this.GetType().Name}");
                _Location = value;
            }
        }
        public string Note { get; set; }
        public bool Assembled { get; set; }

        public bool PackingList { get; set; }

        [Backlink(nameof(JobItem.Job))]
        internal IQueryable<JobItem> JobItems { get; }
        private Job()
        {
        }
        public Job(OrderDetail orderDetail, string jobId)
        {
            this.OrderDetail = orderDetail;
            JobId = jobId;
            JobType = DataService.DefaultJobType();
            Location = DataService.DefaultLocation();
        }
    }
}

filter = "OrderDetail.OrderDetailId == '75edd946-2681-40a8-b83b-f1e66423c30a'"

The property values of rOrderDetailWithoutJob when the exception is thrown is shown in the image

OrderDetail.Job == null is valid

In Realm Studio I can see that for OrderDetail 75edd946-2681-40a8-b83b-f1e66423c30a, Job is null
and there is no job with OrderDetailId 75edd946-2681-40a8-b83b-f1e66423c30a

The data is consistent with the design.

[nirinchev]

Hm, this looks like the issue that was fixed here: https://github.com/realm/realm-core/blob/master/CHANGELOG.md#6025-release-notes. Unfortunately, .NET 5.0.1 is already running Core 6.0.25, which seems to indicate, it's a new undiscovered problem.

[jedelbo]

So the realm file that exhibit this behavior, is that generated with SDK 5.01 or generated with SDK 4.3.0 and then upgraded? In any case it could be very helpful, if you could share a realm file that exposes this error. You can share it privately by sending it to jorgen.edelbo@mongodb.com.

[bmunkholm]

@VagueGit Can you share the realm file?

[VagueGit]

Address of Realm file emailed to @jedelbo

[jedelbo]

Root cause found. We will release a fix as soon as possible.

[tipa]

Still crashes in my UWP app with "External component has thrown an exception. (Exception = {"Error HRESULT E_FAIL has been returned from a call to a COM component."})"

Edit: was testing NuGet v5.1.0 which has just been released and I therefore thought it included the fix

[nirinchev]

This has been fixed in core but hasn't been released for .NET yet.

[hyouuu]

I use RealmSwift on iOS, and when accessing a filtered Results count property e.g. searchResults.count it would crash sometimes with Crashlytics reporting "Key not found", so I think it's probably from the same cause - could you confirm and please let me know if it's not. Thanks!

[hyouuu]

@tipa is this fixed for you? Seems still not fixed on my app with 5.4.7

[hyouuu]

Also cc @VagueGit

[tipa]

Jup, still crashing with Nuget 5.1.1 even though the changelog mentions bugfixes in Core

[hyouuu]

@jedelbo could we reopen this issue and try to find a fix soon? It's causing 10% of my users experiencing crashes which is terrible :(

[hyouuu]

Also, would use v10 fix the issue? https://github.com/realm/realm-cocoa/releases/tag/v10.0.0-rc.1

[VagueGit]

The exception is no longer thrown on the test database where we first struck the error. Thank you @nirinchev @jedelbo

[tipa]

Well, I seem to have a unrelated problem then. I will try to create a repro and open a new issue. Seems like a threading issue that has been introduced with v5.x