Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for FFI while leaving legacy implementations intact #11483

Closed
wants to merge 58 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
394321f
branch to support FFI
JasonLunn Oct 31, 2022
11fcd23
restore protobuf.rb
JasonLunn Oct 31, 2022
09a7114
native implementation
JasonLunn Oct 31, 2022
1e50c8b
restore protobuf.rb
JasonLunn Oct 31, 2022
6cc806c
Refactor `protobuf.rb` to separate native and ffi implementations.
JasonLunn Oct 31, 2022
dfc7dbb
Fix spelling `Unsupport` -> `Unsupported`
JasonLunn Jan 6, 2023
fb395f8
Fix exception raising by properly instantiating the exception.
JasonLunn Jan 6, 2023
d9767ca
Add check on `PROTOCOL_BUFFERS_RUBY_IMPLEMENTATION` environment varia…
JasonLunn Jan 6, 2023
3792576
Add FFI implementation
JasonLunn Jan 6, 2023
589629d
Add ffi flags
mkruskal-google Jan 10, 2023
2ec3aa3
Refactor convert.c to separate CRuby-specific glue that has dependenc…
JasonLunn Jan 11, 2023
a5ff141
Fix copy/paste casting bug.
JasonLunn Jan 11, 2023
cec7850
Refactor message.c to separate CRuby-specific glue that has dependenc…
JasonLunn Jan 11, 2023
4734f34
Have shared_convert methods invoke shared_message methods directly.
JasonLunn Jan 11, 2023
55cd7fe
branch to support FFI
JasonLunn Oct 31, 2022
2f3838b
restore protobuf.rb
JasonLunn Oct 31, 2022
c8656c4
native implementation
JasonLunn Oct 31, 2022
1c19455
restore protobuf.rb
JasonLunn Oct 31, 2022
f461e72
Refactor `protobuf.rb` to separate native and ffi implementations.
JasonLunn Oct 31, 2022
1b1e64e
Fix spelling `Unsupport` -> `Unsupported`
JasonLunn Jan 6, 2023
4cbcd56
Fix exception raising by properly instantiating the exception.
JasonLunn Jan 6, 2023
5d71437
Stop ignoring shared source files; ignore FFI Compiler output directo…
JasonLunn Jul 6, 2023
b402119
Update function names and arguments.
JasonLunn Jul 6, 2023
f7083f1
Remove references to `PROTOCOL_BUFFERS_RUBY_IMPLEMENTATION`.
JasonLunn Jul 6, 2023
039af51
Relocate shared files.
JasonLunn Jul 6, 2023
8607cda
Temporarily update visibility, pending update to upb repo.
JasonLunn Jul 6, 2023
1a716c0
Create glue.
JasonLunn Jul 6, 2023
e7240f8
Fix unit tests.
JasonLunn Jul 7, 2023
8f95755
Rebase
JasonLunn Oct 31, 2022
1dff209
restore protobuf.rb
JasonLunn Oct 31, 2022
f4c08b2
Rebase
JasonLunn Oct 31, 2022
e32280e
restore protobuf.rb
JasonLunn Oct 31, 2022
536f9b3
Rebase
JasonLunn Oct 31, 2022
c665a79
Fix spelling `Unsupport` -> `Unsupported`
JasonLunn Jan 6, 2023
78a2928
Fix exception raising by properly instantiating the exception.
JasonLunn Jan 6, 2023
a7aa7aa
Rebase
JasonLunn Jan 6, 2023
bcc6f6c
Rebase
JasonLunn Jan 11, 2023
8336d0a
Refactor message.c to separate CRuby-specific glue that has dependenc…
JasonLunn Jan 11, 2023
88f1506
Have shared_convert methods invoke shared_message methods directly.
JasonLunn Jan 11, 2023
3089818
Fix merge damage
JasonLunn Jul 7, 2023
95c4535
Call `upb_Arena_Malloc()` directly.
JasonLunn Jul 7, 2023
0286c71
Address PR feedback.
JasonLunn Jul 7, 2023
2fd1eac
Cleanup debug print.
JasonLunn Jul 7, 2023
3d1d1fe
Revert changes to ruby-upb.h.
JasonLunn Jul 10, 2023
0d575d5
Merge remote-tracking branch 'upstream/main' into simultaneous_ffi
JasonLunn Jul 10, 2023
8920f9d
Update to the new Ruby-native ObjectCache.
JasonLunn Jul 10, 2023
72d6ee8
Allow fallback to native CRuby implementation when FFI doesn't load.
JasonLunn Jul 10, 2023
a3eb317
Introduce a new test that the expected implementation is used.
JasonLunn Jul 12, 2023
bc87afb
Merge remote-tracking branch 'upstream/main' into simultaneous_ffi
JasonLunn Jul 12, 2023
ca80881
Merge branch 'main' into simultaneous_ffi
JasonLunn Jul 13, 2023
2c8d0ef
Use the FFI-fix branch of rules_ruby.
JasonLunn Jul 14, 2023
2a1cf7c
Use backwards compatible range syntax.
JasonLunn Jul 14, 2023
e28e256
Move most calls to `attach_function` out of `ffi.rb`.
JasonLunn Jul 15, 2023
55f2884
Breakout conformance tests by platform and whether FFI is enabled.
JasonLunn Jul 15, 2023
03a32da
Merge remote-tracking branch 'upstream/main' into simultaneous_ffi
JasonLunn Jul 15, 2023
472b653
Remove workaround for access to UPI API functions.
JasonLunn Jul 16, 2023
af40b04
Default JRuby to autodetecting FFI but make CRuby request it explicitly.
JasonLunn Jul 17, 2023
839a3d7
Make all interpreters opt-in for FFI.
JasonLunn Jul 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Refactor message.c to separate CRuby-specific glue that has dependenc…
…ies on Ruby headers from functionality needed by FFI.
  • Loading branch information
JasonLunn committed Jul 7, 2023
commit 8336d0aedbc189088c3edbefc439f7ba49b8512f
87 changes: 87 additions & 0 deletions ruby/ext/google/shared/shared_message.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2023 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

// -----------------------------------------------------------------------------
// Ruby Message functions. Strictly free of dependencies on
// Ruby interpreter internals.

#include "shared_message.h"

// Support function for Message_Hash. Returns a hash value for the given message.
uint64_t shared_Message_Hash(const upb_Message* msg, const upb_MessageDef* m,
uint64_t seed, upb_Status* status) {
upb_Arena* arena = upb_Arena_New();
char* data;
size_t size;

// Hash a deterministically serialized payloads with no unknown fields.
upb_EncodeStatus encode_status = upb_Encode(
msg, upb_MessageDef_MiniTable(m),
kUpb_EncodeOption_SkipUnknown | kUpb_EncodeOption_Deterministic, arena,
&data, &size);

if (encode_status == kUpb_EncodeStatus_Ok) {
uint64_t ret = _upb_Hash(data, size, seed);
upb_Arena_Free(arena);
return ret;
} else {
upb_Arena_Free(arena);
upb_Status_SetErrorMessage(status, "Error calculating hash");
}
}

// Support function for Message_Equal
bool shared_Message_Equal(const upb_Message* m1, const upb_Message* m2,
const upb_MessageDef* m, upb_Status* status) {
if (m1 == m2) return true;

size_t size1, size2;
int encode_opts =
kUpb_EncodeOption_SkipUnknown | kUpb_EncodeOption_Deterministic;
upb_Arena* arena_tmp = upb_Arena_New();
const upb_MiniTable* layout = upb_MessageDef_MiniTable(m);

// Compare deterministically serialized payloads with no unknown fields.
char* data1;
char* data2;
upb_EncodeStatus status1 =
upb_Encode(m1, layout, encode_opts, arena_tmp, &data1, &size1);
upb_EncodeStatus status2 =
upb_Encode(m2, layout, encode_opts, arena_tmp, &data2, &size2);

if (status1 == kUpb_EncodeStatus_Ok && status2 == kUpb_EncodeStatus_Ok) {
bool ret = (size1 == size2) && (memcmp(data1, data2, size1) == 0);
upb_Arena_Free(arena_tmp);
return ret;
} else {
upb_Arena_Free(arena_tmp);
upb_Status_SetErrorMessage(status, "Error comparing messages");
}
}
48 changes: 48 additions & 0 deletions ruby/ext/google/shared/shared_message.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2023 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

// -----------------------------------------------------------------------------
// Ruby Message functions. Strictly free of dependencies on
// Ruby interpreter internals.

#ifndef RUBY_PROTOBUF_SHARED_MESSAGE_H_
#define RUBY_PROTOBUF_SHARED_MESSAGE_H_

#include "ruby-upb.h"

// Returns a hash value for the given message.
uint64_t shared_Message_Hash(const upb_Message* msg, const upb_MessageDef* m,
uint64_t seed, upb_Status* status);

// Returns true if these two messages are equal.
bool shared_Message_Equal(const upb_Message* m1, const upb_Message* m2,
const upb_MessageDef* m, upb_Status* status);

#endif // RUBY_PROTOBUF_SHARED_MESSAGE_H_