Ractor support

[sandlerr]

Hello maintainers,

Is there any interest or planned change to add support to this gem for use within a Ruby 3 Ractor?

Currently, constructing a database instance is failing with the error message introduced into this Ruby commit. I have collected this stacktrace:

~/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:89:in `open_v2': ractor unsafe method called from not main ractor (Ractor::UnsafeError)
        from ~/.rbenv/versions/3.0.1/lib/ruby/gems/3.0.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:89:in `initialize'
        ...

It can be reproduced by:

sqlite_ractor = Ractor.new do
    db = SQLite3::Database.open "xyz.db"
end
puts sqlite_ractor.take

The Ruby commit message justifies the error because:

C extensions can violate the ractor-safety

However, this gem doesn't appear to store references to Ruby objects that might be accessed across Ractors / invalidly shared, so perhaps its C extensions can be marked as Ractor-safe?

[tenderlove]

Hi,

I would like to support Ractors, but I'm not sure how we would do that exactly. SQLite3 is a C library, so we must make calls to C functions and as such this is a Ruby C extension. If there is a way to make C extensions play nicely with Ractors, then I'm all for it! 😄

[sandlerr]

@tenderlove , thank you for your feedback from last year. It can sort of be done, but unfortunately not yet to the point where one could conceivably build something like a connection pool - not that this is a useful thing for SQLite users, but it could be a starting point for other projects exploring ractors.

Basic support and testing was built by @KJTsanaktsidis and I've included it with commented details on the limitations being tested here: master...sandlerr:sqlite3-ruby:sandlerr/ractor

If there is a way to make C extensions play nicely with Ractors, then I'm all for it!

This part is a single C API call (after eyeballing some code and not seeing obvious violations of the ractor rules, like storing VALUEs that might be accessed incorrectly in another thread).

[tenderlove]

@sandlerr tbh that patch looks fine to me. We could add it and then tackle any issues as they come up. It looks like the only issue in the patch was the lambda?

[sandlerr]

Okay, I'm working on a version that compiles under Ruby 2, hopefully using weak attributes so the extension can be built under Ruby 2 or 3 and in either case will work correctly under both Ruby 2 and 3. The current iteration of the patch just assumes that ractors exist, so fails to compile in Ruby 2.

I imagine publishing separate prebuilt versions to rubygems for Ruby 2 and 3 is undesirable, but please let me know if this seems misguided.

It looks like the only issue in the patch was the lambda?

This was the only blocker to creating a DB and making simple insert queries, yes.

[flavorjones]

@sandlerr compile-time support should be fine.

The current convention is that each native gem file contains multiple precompiled extensions, one per each version of ruby supported. So, for example, the installed files in sqlite3-1.5.4-x86_64-linux.gem are:

lib
├── sqlite3
│   ├── 2.6
│   │   └── sqlite3_native.so
│   ├── 2.7
│   │   └── sqlite3_native.so
│   ├── 3.0
│   │   └── sqlite3_native.so
│   ├── 3.1
│   │   └── sqlite3_native.so
│   ├── constants.rb
│   ├── database.rb
│   ├── errors.rb
│   ├── pragmas.rb
│   ├── resultset.rb
│   ├── statement.rb
│   ├── translator.rb
│   ├── value.rb
│   └── version.rb
└── sqlite3.rb

and only the dynamic library corresponding to the version of Ruby at runtime is loaded. Make sense?

[sandlerr]

Yes, thanks for the pointer. #365