[perf] upgrade realm and improve schemas/queries #8336

yenda · 2019-06-02T02:29:38Z

upgrade to realm 2.28 to benefit from perf improvements
remove user-statuses and replace by seen and outgoing-status fields
to get rid of a lot of bloat queries and computations and in-queries which don't
work in this version of realm
remove unused seen message, bottom-infos

Tests

Tested on Android

Tested:

send and receive messages from public and private chats

To test:

start from nightlies with some chats that have unread messages and unsent messages
install this version and check that the migrations worked

status: wip

status-github-bot · 2019-06-02T02:29:44Z

Pull Request Checklist

Have you updated the documentation, if impacted (e.g. docs.status.im)?

status-im-auto · 2019-06-02T02:43:09Z

Jenkins Builds

Click to see older builds (24)

❔	Commit	#️⃣	Finished (UTC)	Duration	Platform	Result
✔️	`d3294d6`	#1	2019-06-02 02:43:09	~13 min	`ios`	📦 ipa
✔️	`d3294d6`	#1	2019-06-02 02:44:37	~14 min	`linux`	📦 App
✔️	`d3294d6`	#1	2019-06-02 02:45:32	~15 min	`macos`	📦 dmg
✔️	`d3294d6`	#1	2019-06-02 02:45:44	~15 min	`windows`	📦 exe
✔️	`d3294d6`	#1	2019-06-02 02:47:40	~17 min	`android-e2e`	📦 apk
✔️	`d3294d6`	#1	2019-06-02 02:48:37	~18 min	`android`	📦 apk

✔️	`458543a`	#2	2019-06-03 16:45:11	~13 min	`linux`	📦 App
✔️	`458543a`	#2	2019-06-03 16:45:57	~14 min	`ios`	📦 ipa
✔️	`458543a`	#2	2019-06-03 16:46:25	~14 min	`windows`	📦 exe
✔️	`458543a`	#2	2019-06-03 16:49:17	~17 min	`macos`	📦 dmg
✔️	`458543a`	#2	2019-06-03 16:50:44	~18 min	`android`	📦 apk
❌	`458543a`	#2	2019-06-03 16:54:58	~23 min	`android-e2e`	📄 log

❌	`4695338`	#3	2019-06-04 10:55:24	~3 min	`ios`	📄 log
❌	`4695338`	#3	2019-06-04 10:55:31	~3 min	`linux`	📄 log
❌	`4695338`	#3	2019-06-04 10:55:41	~3 min	`windows`	📄 log
❌	`4695338`	#3	2019-06-04 10:56:00	~3 min	`macos`	📄 log
❌	`4695338`	#3	2019-06-04 11:01:57	~9 min	`android`	📄 log
❌	`4695338`	#3	2019-06-04 11:02:01	~9 min	`android-e2e`	📄 log

✔️	`4974141`	#4	2019-06-04 11:18:41	~15 min	`ios`	📦 ipa
✔️	`4974141`	#4	2019-06-04 11:19:28	~15 min	`windows`	📦 exe
✔️	`4974141`	#4	2019-06-04 11:19:28	~15 min	`linux`	📦 App
✔️	`4974141`	#4	2019-06-04 11:20:04	~16 min	`macos`	📦 dmg
✔️	`4974141`	#4	2019-06-04 11:23:39	~20 min	`android`	📦 apk
✔️	`4974141`	#4	2019-06-04 11:24:27	~20 min	`android-e2e`	📦 apk

❔	Commit	#️⃣	Finished (UTC)	Duration	Platform	Result
✔️	`5035895`	#5	2019-06-04 11:55:06	~13 min	`ios`	📦 ipa
✔️	`5035895`	#5	2019-06-04 11:55:39	~13 min	`linux`	📦 App
✔️	`5035895`	#5	2019-06-04 11:56:14	~14 min	`windows`	📦 exe
✔️	`5035895`	#5	2019-06-04 12:00:37	~18 min	`macos`	📦 dmg
✔️	`5035895`	#5	2019-06-04 12:01:33	~19 min	`android`	📦 apk
✔️	`5035895`	#5	2019-06-04 12:01:44	~20 min	`android-e2e`	📦 apk

✔️	`fda6c36`	#7	2019-06-04 21:49:36	~13 min	`ios`	📦 ipa
✔️	`fda6c36`	#7	2019-06-04 21:50:38	~14 min	`android`	📦 apk
✔️	`fda6c36`	#7	2019-06-04 21:52:04	~16 min	`macos`	📦 dmg
✔️	`fda6c36`	#7	2019-06-04 21:53:21	~17 min	`android-e2e`	📦 apk
✔️	`fda6c36`	#7	2019-06-04 21:54:09	~18 min	`windows`	📦 exe
✔️	`fda6c36`	#7	2019-06-04 21:54:49	~18 min	`linux`	📦 App

jakubgs · 2019-06-03T16:05:28Z

I wonder if this might help with #8326 .

jakubgs

I'm approving only the yarn.lock changed, the Clojure stuff is black magic.

flexsurfer · 2019-06-03T18:05:31Z

src/status_im/chat/models/message.cljs

-      (assoc :appearing? false)
+    (cond-> message
+      current-chat?
+      (assoc :seen true)


could you elaborate?

appearing is useless, not needed.
If the message is received in the current chat it is considered seen. It was already the case before just not done here

statustestbot · 2019-06-04T13:27:42Z

98% of end-end tests have passed

Total executed tests: 49
Failed tests: 1
Passed tests: 48

Failed tests (1)

Click to expand

1. test_send_tokens_in_1_1_chat
Device 2: Looking for a message by text: '0.004131042' Device 2: Wait for ChatElementByText `Message with the sent amount is not shown for the recipient, also Unexpected Alert is shown: 'Array size is not a small enough positive integer.'` Device sessions Device 2: Steps, video, logs Failure screenshot Device 1: Steps, video, logs Failure screenshot

Passed tests (48)

Click to expand

1. test_block_user_from_public_chat
Device sessions Device 2: Steps, video, logs Device 1: Steps, video, logs
2. test_filters_from_daap
Device sessions Device 1: Steps, video, logs
3. test_copy_and_paste_messages
Device sessions Device 1: Steps, video, logs
4. test_send_transaction_from_daap
Device sessions Device 1: Steps, video, logs
5. test_request_and_receive_tokens_in_1_1_chat
Device sessions Device 2: Steps, video, logs Device 1: Steps, video, logs
6. test_deploy_contract_from_daap
Device sessions Device 1: Steps, video, logs
7. test_open_transaction_on_etherscan
Device sessions Device 1: Steps, video, logs
8. test_public_chat_messaging
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
9. test_long_press_to_delete_1_1_chat
Device sessions Device 1: Steps, video, logs
10. test_password_in_logcat_sign_in
Device sessions Device 1: Steps, video, logs
11. test_text_message_1_1_chat
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
12. test_add_to_contacts
Device sessions Device 2: Steps, video, logs Device 1: Steps, video, logs
13. test_sign_typed_message (TestRail link is not found)
Device sessions Device 1: Steps, video, logs
14. test_unread_messages_counter_1_1_chat
Device sessions Device 2: Steps, video, logs Device 1: Steps, video, logs
15. test_logcat_send_transaction_from_daap
Device sessions Device 1: Steps, video, logs
16. test_send_message_in_group_chat
Device sessions Device 2: Steps, video, logs Device 1: Steps, video, logs
17. test_logcat_send_transaction_from_wallet
Device sessions Device 1: Steps, video, logs
18. test_send_token_with_7_decimals
Device sessions Device 1: Steps, video, logs
19. test_modify_transaction_fee_values
Device sessions Device 1: Steps, video, logs
20. test_send_eth_from_wallet_to_address
Device sessions Device 1: Steps, video, logs
21. test_manage_assets
Device sessions Device 1: Steps, video, logs
22. test_logcat_send_transaction_in_1_1_chat
Device sessions Device 1: Steps, video, logs
23. test_request_and_receive_eth_in_1_1_chat
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
24. test_long_press_to_delete_public_chat
Device sessions Device 1: Steps, video, logs
25. test_send_emoji
Device sessions Device 1: Steps, video, logs
26. test_search_chat_on_home
Device sessions Device 1: Steps, video, logs
27. test_logcat_recovering_account
Device sessions Device 1: Steps, video, logs
28. test_messaging_in_different_networks
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
29. test_network_mismatch_for_send_request_commands
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
30. test_logcat_sign_message_from_daap
Device sessions Device 1: Steps, video, logs
31. test_switch_users_and_add_new_account
Device sessions Device 1: Steps, video, logs
32. test_send_stt_from_wallet
Device sessions Device 1: Steps, video, logs
33. test_send_eth_in_1_1_chat
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
34. test_login_with_new_account
Device sessions Device 1: Steps, video, logs
35. test_send_eth_from_wallet_to_contact
Device sessions Device 1: Steps, video, logs
36. test_add_contact_from_public_chat
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
37. test_send_two_transactions_one_after_another_in_dapp
Device sessions Device 1: Steps, video, logs
38. test_password_in_logcat_creating_account
Device sessions Device 1: Steps, video, logs
39. test_backup_recovery_phrase
Device sessions Device 1: Steps, video, logs
40. test_offline_status
Device sessions Device 1: Steps, video, logs
41. test_open_google_com_via_open_dapp
Device sessions Device 1: Steps, video, logs
42. test_unread_messages_counter_public_chat
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
43. test_sign_message_from_daap
Device sessions Device 1: Steps, video, logs
44. test_user_can_remove_profile_picture
Device sessions Device 1: Steps, video, logs
45. test_share_contact_code_and_wallet_address
Device sessions Device 1: Steps, video, logs
46. test_request_eth_in_wallet
Device sessions Device 1: Steps, video, logs Device 2: Steps, video, logs
47. test_refresh_button_browsing_app_webview
Device sessions Device 1: Steps, video, logs
48. test_backup_recovery_phrase_warning_from_wallet
Device sessions Device 1: Steps, video, logs

statustestbot · 2019-06-04T14:50:31Z

100% of end-end tests have passed

Total executed tests: 1
Failed tests: 0
Passed tests: 1

Passed tests (1)

Click to expand

1. test_send_tokens_in_1_1_chat
Device sessions Device 2: Steps, video, logs Device 1: Steps, video, logs

yenda · 2019-06-04T15:16:06Z

@asemiankevich this is a bug I've seen a few times already in other PR, I have never been able to reproduce in the emulator so I don't really now where it is coming from

dependency-lockfile-snitch · 2019-06-04T15:16:50Z

mobile_files/yarn.lock changed. Pinging @jakubgs, @mandrigin, and @corpetty

dependency-lockfile-snitch · 2019-06-04T15:18:32Z

mobile_files/yarn.lock changed. Pinging @jakubgs, @mandrigin, and @corpetty

asemiankevich · 2019-06-04T18:44:15Z

@asemiankevich this is a bug I've seen a few times already in other PR, I have never been able to reproduce in the emulator so I don't really now where it is coming from

@yenda i've logged it separately, no worries! Apart from this current PR looks good to me, upgrade looks okay-ish.

dependency-lockfile-snitch · 2019-06-04T21:33:50Z

mobile_files/yarn.lock changed. Pinging @jakubgs, @mandrigin, and @corpetty

dependency-lockfile-snitch · 2019-06-04T21:35:42Z

mobile_files/yarn.lock changed. Pinging @jakubgs, @mandrigin, and @corpetty

- upgrade to realm 2.28 to benefit from perf improvements - remove user-statuses and replace by seen and outgoing-status fields to get rid of a lot of bloat queries and computations - remove unused seen message, bottom-infos - remove unused fields in transport schema - use objectForPrimaryKey whenever possible instead of get by field Signed-off-by: yenda <eric@status.im>

dependency-lockfile-snitch · 2019-06-04T21:37:01Z

mobile_files/yarn.lock changed. Pinging @jakubgs, @mandrigin, and @corpetty

yenda requested review from cammellos, vitvly, rasom and flexsurfer June 2, 2019 02:29

yenda requested review from antdanchenko, churik, vkjr and a team as code owners June 2, 2019 02:29

yenda self-assigned this Jun 2, 2019

yenda mentioned this pull request Jun 3, 2019

Unread count still does not disappear after messages are sent #7497

Closed

mandrigin approved these changes Jun 3, 2019

View reviewed changes

yenda force-pushed the perf/upgrade-realm-and-schemas branch from d3294d6 to 458543a Compare June 3, 2019 16:31

jakubgs approved these changes Jun 3, 2019

View reviewed changes

flexsurfer approved these changes Jun 3, 2019

View reviewed changes

yenda force-pushed the perf/upgrade-realm-and-schemas branch from 4695338 to 4974141 Compare June 4, 2019 11:03

yenda force-pushed the perf/upgrade-realm-and-schemas branch from 4974141 to 5035895 Compare June 4, 2019 11:41

status-im deleted a comment from statustestbot Jun 4, 2019

status-im deleted a comment from dependency-lockfile-snitch bot Jun 4, 2019

status-im deleted a comment from statustestbot Jun 4, 2019

status-im deleted a comment from dependency-lockfile-snitch bot Jun 4, 2019

asemiankevich self-assigned this Jun 4, 2019

asemiankevich added the Tested - OK label Jun 4, 2019

yenda force-pushed the perf/upgrade-realm-and-schemas branch from 5035895 to af0e10d Compare June 4, 2019 21:33

yenda force-pushed the perf/upgrade-realm-and-schemas branch from af0e10d to fda6c36 Compare June 4, 2019 21:35

yenda force-pushed the perf/upgrade-realm-and-schemas branch from fda6c36 to 1ae42ea Compare June 4, 2019 21:36

yenda merged commit 1ae42ea into develop Jun 4, 2019

delete-merged-branch bot deleted the perf/upgrade-realm-and-schemas branch June 4, 2019 21:37

jakubgs mentioned this pull request Jun 4, 2019

don't exclude arm64-v8a libraries #8326

Closed

annadanchenko mentioned this pull request Jun 5, 2019

Can't launch mobile app after realm upgrade: Unfortunately, Status has stopped (Android 6.0.1, Galaxy S6 and S7) #8366

Closed

weekly-digest bot mentioned this pull request Jun 9, 2019

Weekly Digest (2 June, 2019 - 9 June, 2019) #8389

Closed

cammellos mentioned this pull request Jun 17, 2019

Partitioned topic and topic negotiation #8363

Merged

StatusWrike unassigned asemiankevich Sep 12, 2019

StatusWrike unassigned yenda Apr 29, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[perf] upgrade realm and improve schemas/queries #8336

[perf] upgrade realm and improve schemas/queries #8336

yenda commented Jun 2, 2019 •

edited by StatusWrike

Loading

status-github-bot bot commented Jun 2, 2019 •

edited by yenda

Loading

status-im-auto commented Jun 2, 2019 •

edited

Loading

jakubgs commented Jun 3, 2019

jakubgs left a comment

flexsurfer Jun 3, 2019

yenda Jun 3, 2019

statustestbot commented Jun 4, 2019

statustestbot commented Jun 4, 2019

yenda commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

asemiankevich commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

[perf] upgrade realm and improve schemas/queries #8336

[perf] upgrade realm and improve schemas/queries #8336

Conversation

yenda commented Jun 2, 2019 • edited by StatusWrike Loading

Tests

status-github-bot bot commented Jun 2, 2019 • edited by yenda Loading

Pull Request Checklist

status-im-auto commented Jun 2, 2019 • edited Loading

Jenkins Builds

jakubgs commented Jun 3, 2019

jakubgs left a comment

Choose a reason for hiding this comment

flexsurfer Jun 3, 2019

Choose a reason for hiding this comment

yenda Jun 3, 2019

Choose a reason for hiding this comment

statustestbot commented Jun 4, 2019

98% of end-end tests have passed

Failed tests (1)

Passed tests (48)

statustestbot commented Jun 4, 2019

100% of end-end tests have passed

Passed tests (1)

yenda commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

asemiankevich commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

dependency-lockfile-snitch bot commented Jun 4, 2019

yenda commented Jun 2, 2019 •

edited by StatusWrike

Loading

status-github-bot bot commented Jun 2, 2019 •

edited by yenda

Loading

status-im-auto commented Jun 2, 2019 •

edited

Loading