From a52bbf87dbd788759855394a3ce2a66ec6b50087 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 10:02:21 -0500 Subject: [PATCH 01/50] Fix broken link in changelog --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a38b111..4a7671a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -100,7 +100,7 @@ All notable changes to this project will be documented in this file. This change - Support for serializing and deserializing arbitrary Clojure functions - Support for serializing and deserializing arbitrary Clojure data structures -[Unreleased]: https://github.com/IGJoshua/coffi/compare/v0.4.341...develop +[0.5.357]: https://github.com/IGJoshua/coffi/compare/v0.4.341...v0.5.357 [0.4.341]: https://github.com/IGJoshua/coffi/compare/v0.3.298...v0.4.341 [0.3.298]: https://github.com/IGJoshua/coffi/compare/v0.2.277...v0.3.298 [0.2.277]: https://github.com/IGJoshua/coffi/compare/v0.2.259...v0.2.277 From f2c1d30c8b3fef956fad7feface58310ed1450ae Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 10:02:32 -0500 Subject: [PATCH 02/50] Add unreleased section to changelog --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4a7671a..733d9e4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,8 @@ # Change Log All notable changes to this project will be documented in this file. This change log follows the conventions of [keepachangelog.com](http://keepachangelog.com/). +## [Unreleased] + ## [0.5.357] - 2022-07-07 ### Removed - `:coffi.mem/long-long` primitive type @@ -100,6 +102,7 @@ All notable changes to this project will be documented in this file. This change - Support for serializing and deserializing arbitrary Clojure functions - Support for serializing and deserializing arbitrary Clojure data structures +[Unreleased]: https://github.com/IGJoshua/coffi/compare/v0.5.357...develop [0.5.357]: https://github.com/IGJoshua/coffi/compare/v0.4.341...v0.5.357 [0.4.341]: https://github.com/IGJoshua/coffi/compare/v0.3.298...v0.4.341 [0.3.298]: https://github.com/IGJoshua/coffi/compare/v0.2.277...v0.3.298 From e98c5a56bb4699035ebf3b3b49a8e89f8edbe087 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 10:05:30 -0500 Subject: [PATCH 03/50] Update codox version info --- deps.edn | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/deps.edn b/deps.edn index be56f5b..298cdeb 100644 --- a/deps.edn +++ b/deps.edn @@ -24,8 +24,8 @@ :codox {:extra-deps {codox/codox {:mvn/version "0.10.7"}} :exec-fn codox.main/generate-docs :exec-args {:name "coffi" - :version "v0.4.341" - :description "A Foreign Function Interface in Clojure for JDK 17." + :version "v0.5.357" + :description "A Foreign Function Interface in Clojure for JDK 18." :source-paths ["src/clj"] :output-path "docs" :source-uri "https://github.com/IGJoshua/coffi/blob/{git-commit}/{filepath}#L{line}" From c3b612fdb0f77711d8de63073f64c2dcf3fe16b0 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 10:06:02 -0500 Subject: [PATCH 04/50] Update codox --- docs/coffi.ffi.html | 18 +++++------ docs/coffi.layout.html | 4 +-- docs/coffi.mem.html | 70 +++++++++++++++++++++--------------------- docs/index.html | 2 +- 4 files changed, 47 insertions(+), 47 deletions(-) diff --git a/docs/coffi.ffi.html b/docs/coffi.ffi.html index a8fea82..ac3e8c1 100644 --- a/docs/coffi.ffi.html +++ b/docs/coffi.ffi.html @@ -1,19 +1,19 @@ -coffi.ffi documentation

Project

Namespaces

Public Vars

coffi.ffi

Functions for creating handles to native functions and loading native libraries.

cfn

(cfn symbol args ret)

Constructs a Clojure function to call the native function referenced by symbol.

+coffi.ffi documentation

Project

Namespaces

Public Vars

coffi.ffi

Functions for creating handles to native functions and loading native libraries.

cfn

(cfn symbol args ret)

Constructs a Clojure function to call the native function referenced by symbol.

The function returned will serialize any passed arguments into the args types, and deserialize the return to the ret type.

-

If your args and ret are constants, then it is more efficient to call make-downcall followed by make-serde-wrapper because the latter has an inline definition which will result in less overhead from serdes.

view source

const

(const symbol-or-addr type)

Gets the value of a constant stored in symbol-or-addr.

view source

defcfn

macro

(defcfn name docstring? attr-map? symbol arg-types ret-type)(defcfn name docstring? attr-map? symbol arg-types ret-type native-fn & fn-tail)

Defines a Clojure function which maps to a native function.

+

If your args and ret are constants, then it is more efficient to call make-downcall followed by make-serde-wrapper because the latter has an inline definition which will result in less overhead from serdes.

view source

const

(const symbol-or-addr type)

Gets the value of a constant stored in symbol-or-addr.

view source

defcfn

macro

(defcfn name docstring? attr-map? symbol arg-types ret-type)(defcfn name docstring? attr-map? symbol arg-types ret-type native-fn & fn-tail)

Defines a Clojure function which maps to a native function.

name is the symbol naming the resulting var. symbol is a symbol or string naming the library symbol to link against. arg-types is a vector of qualified keywords representing the argument types. ret-type is a single qualified keyword representing the return type. fn-tail is the body of the function (potentially with multiple arities) which wraps the native one. Inside the function, native-fn is bound to a function that will serialize its arguments, call the native function, and deserialize its return type. If any body is present, you must call this function in order to call the native code.

If no fn-tail is provided, then the resulting function will simply serialize the arguments according to arg-types, call the native function, and deserialize the return value.

The number of args in the fn-tail need not match the number of arg-types for the native function. It need only call the native wrapper function with the correct arguments.

-

See serialize, deserialize, make-downcall.

view source

find-symbol

(find-symbol sym)

Gets the NativeSymbol of a symbol from the loaded libraries.

view source

freset!

(freset! static-var newval)

Sets the value of static-var to newval, running it through serialize.

view source

fswap!

(fswap! static-var f & args)

Non-atomically runs the function f over the value stored in static-var.

-

The value is deserialized before passing it to f, and serialized before putting the value into static-var.

view source

load-library

(load-library path)

Loads the library at path.

view source

load-system-library

(load-system-library libname)

Loads the library named libname from the system’s load path.

view source

make-downcall

(make-downcall symbol-or-addr args ret)

Constructs a downcall function reference to symbol-or-addr with the given args and ret types.

+

See serialize, deserialize, make-downcall.

view source

find-symbol

(find-symbol sym)

Gets the NativeSymbol of a symbol from the loaded libraries.

view source

freset!

(freset! static-var newval)

Sets the value of static-var to newval, running it through serialize.

view source

fswap!

(fswap! static-var f & args)

Non-atomically runs the function f over the value stored in static-var.

+

The value is deserialized before passing it to f, and serialized before putting the value into static-var.

view source

load-library

(load-library path)

Loads the library at path.

view source

load-system-library

(load-system-library libname)

Loads the library named libname from the system’s load path.

view source

make-downcall

(make-downcall symbol-or-addr args ret)

Constructs a downcall function reference to symbol-or-addr with the given args and ret types.

The function returned takes only arguments whose types match exactly the java-layout for that type, and returns an argument with exactly the java-layout of the ret type. This function will perform no serialization or deserialization of arguments or the return type.

-

If the ret type is non-primitive, then the returned function will take a first argument of a SegmentAllocator.

view source

make-serde-varargs-wrapper

(make-serde-varargs-wrapper varargs-factory required-args ret-type)

Constructs a wrapper function for the varargs-factory which produces functions that serialize the arguments and deserialize the return value.

view source

make-serde-wrapper

(make-serde-wrapper downcall arg-types ret-type)

Constructs a wrapper function for the downcall which serializes the arguments and deserializes the return value.

view source

make-varargs-factory

(make-varargs-factory symbol required-args ret)

Returns a function for constructing downcalls with additional types for arguments.

+

If the ret type is non-primitive, then the returned function will take a first argument of a SegmentAllocator.

view source

make-serde-varargs-wrapper

(make-serde-varargs-wrapper varargs-factory required-args ret-type)

Constructs a wrapper function for the varargs-factory which produces functions that serialize the arguments and deserialize the return value.

view source

make-serde-wrapper

(make-serde-wrapper downcall arg-types ret-type)

Constructs a wrapper function for the downcall which serializes the arguments and deserializes the return value.

view source

make-varargs-factory

(make-varargs-factory symbol required-args ret)

Returns a function for constructing downcalls with additional types for arguments.

The required-args are the types of the first arguments passed to the downcall handle, and the values passed to the returned function are only the varargs types.

The returned function is memoized, so that only one downcall function will be generated per combination of argument types.

-

See make-downcall.

view source

reify-libspec

(reify-libspec libspec)

Loads all the symbols specified in the libspec.

-

The value of each key of the passed map is transformed as by reify-symbolspec.

view source

reify-symbolspec

multimethod

Takes a spec for a symbol reference and returns a live value for that type.

view source

static-variable

(static-variable symbol-or-addr type)

Constructs a reference to a mutable value stored in symbol-or-addr.

+

See make-downcall.

view source

reify-libspec

(reify-libspec libspec)

Loads all the symbols specified in the libspec.

+

The value of each key of the passed map is transformed as by reify-symbolspec.

view source

reify-symbolspec

multimethod

Takes a spec for a symbol reference and returns a live value for that type.

view source

static-variable

(static-variable symbol-or-addr type)

Constructs a reference to a mutable value stored in symbol-or-addr.

The returned value can be dereferenced, and has metadata, and the address of the value can be queried with address-of.

-

See freset!, fswap!.

view source

vacfn-factory

(vacfn-factory symbol required-args ret)

Constructs a varargs factory to call the native function referenced by symbol.

-

The function returned takes any number of type arguments and returns a specialized Clojure function for calling the native function with those arguments.

view source
\ No newline at end of file +

See freset!, fswap!.

view source

vacfn-factory

(vacfn-factory symbol required-args ret)

Constructs a varargs factory to call the native function referenced by symbol.

+

The function returned takes any number of type arguments and returns a specialized Clojure function for calling the native function with those arguments.

view source
\ No newline at end of file diff --git a/docs/coffi.layout.html b/docs/coffi.layout.html index e5aca21..0d699fe 100644 --- a/docs/coffi.layout.html +++ b/docs/coffi.layout.html @@ -1,4 +1,4 @@ -coffi.layout documentation

Project

Namespaces

Public Vars

coffi.layout

Functions for adjusting the layout of structs.

with-c-layout

(with-c-layout struct-spec)

Forces a struct specification to C layout rules.

-

This will add padding fields between fields to match C alignment requirements.

view source
\ No newline at end of file +coffi.layout documentation

Project

Namespaces

Public Vars

coffi.layout

Functions for adjusting the layout of structs.

with-c-layout

(with-c-layout struct-spec)

Forces a struct specification to C layout rules.

+

This will add padding fields between fields to match C alignment requirements.

view source
\ No newline at end of file diff --git a/docs/coffi.mem.html b/docs/coffi.mem.html index 207830a..0378250 100644 --- a/docs/coffi.mem.html +++ b/docs/coffi.mem.html @@ -1,49 +1,49 @@ -coffi.mem documentation

Project

Namespaces

Public Vars

coffi.mem

Functions for managing native allocations, resource scopes, and (de)serialization.

+coffi.mem documentation

Project

Namespaces

Public Vars

coffi.mem

Functions for managing native allocations, resource scopes, and (de)serialization.

For any new type to be implemented, three multimethods must be overriden, but which three depends on the native representation of the type.

If the native representation of the type is a primitive (whether or not other data beyond the primitive is associated with it, as e.g. a pointer), then primitive-type must be overriden to return which primitive type it is serialized as, then serialize* and deserialize* should be overriden.

If the native representation of the type is a composite type, like a union, struct, or array, then c-layout must be overriden to return the native layout of the type, and serialize-into and deserialize-from should be overriden to allow marshaling values of the type into and out of memory segments.

-

When writing code that manipulates a segment, it’s best practice to use with-acquired on the segment-scope in order to ensure it won’t be released during its manipulation.

add-close-action!

(add-close-action! scope action)

Adds a 0-arity function to be run when the scope closes.

view source

address-of

(address-of addressable)

Gets the address of a given segment.

-

This value can be used as an argument to functions which take a pointer.

view source

address?

(address? addr)

Checks if an object is a memory address.

-

nil is considered an address.

view source

align-of

(align-of type)

The alignment in bytes of the given type.

view source

alloc

(alloc size)(alloc size scope)

Allocates size bytes.

-

If a scope is provided, the allocation will be reclaimed when it is closed.

view source

alloc-instance

(alloc-instance type)(alloc-instance type scope)

Allocates a memory segment for the given type.

view source

alloc-with

(alloc-with allocator size)(alloc-with allocator size alignment)

Allocates size bytes using the allocator.

view source

as-segment

(as-segment address size)(as-segment address size scope)

Dereferences an address into a memory segment associated with the scope.

view source

big-endian

The big-endian ByteOrder.

-

See little-endian, native-endian.

view source

byte-layout

The MemoryLayout for a byte in native-endian ByteOrder.

view source

c-layout

multimethod

Gets the layout object for a given type.

+

When writing code that manipulates a segment, it’s best practice to use with-acquired on the segment-scope in order to ensure it won’t be released during its manipulation.

add-close-action!

(add-close-action! scope action)

Adds a 0-arity function to be run when the scope closes.

view source

address-of

(address-of addressable)

Gets the address of a given segment.

+

This value can be used as an argument to functions which take a pointer.

view source

address?

(address? addr)

Checks if an object is a memory address.

+

nil is considered an address.

view source

align-of

(align-of type)

The alignment in bytes of the given type.

view source

alloc

(alloc size)(alloc size scope)

Allocates size bytes.

+

If a scope is provided, the allocation will be reclaimed when it is closed.

view source

alloc-instance

(alloc-instance type)(alloc-instance type scope)

Allocates a memory segment for the given type.

view source

alloc-with

(alloc-with allocator size)(alloc-with allocator size alignment)

Allocates size bytes using the allocator.

view source

as-segment

(as-segment address size)(as-segment address size scope)

Dereferences an address into a memory segment associated with the scope.

view source

big-endian

The big-endian ByteOrder.

+

See little-endian, native-endian.

view source

byte-layout

The MemoryLayout for a byte in native-endian ByteOrder.

view source

c-layout

multimethod

Gets the layout object for a given type.

If a type is primitive it will return the appropriate primitive layout (see c-prim-layout).

-

Otherwise, it should return a GroupLayout for the given type.

view source

char-layout

The MemoryLayout for a c-sized char in native-endian ByteOrder.

view source

clone-segment

(clone-segment segment)(clone-segment segment scope)

Clones the content of segment into a new segment of the same size.

view source

connected-scope

(connected-scope)

Constructs a new scope to reclaim all connected resources at once.

+

Otherwise, it should return a GroupLayout for the given type.

view source

char-layout

The MemoryLayout for a c-sized char in native-endian ByteOrder.

view source

clone-segment

(clone-segment segment)(clone-segment segment scope)

Clones the content of segment into a new segment of the same size.

view source

connected-scope

(connected-scope)

Constructs a new scope to reclaim all connected resources at once.

The scope may be shared across threads, and all resources created with it will be cleaned up at the same time, when all references have been collected.

-

This type of scope cannot be closed, and therefore should not be created in a with-open clause.

view source

copy-segment

(copy-segment dest src)

Copies the content to dest from src.

-

Returns dest.

view source

defalias

macro

(defalias new-type aliased-type)

Defines a type alias from new-type to aliased-type.

-

This creates needed serialization and deserialization implementations for the aliased type.

view source

deserialize

(deserialize obj type)

Deserializes an arbitrary type.

-

For types which have a primitive representation, this deserializes the primitive representation. For types which do not, this deserializes out of a segment.

view source

deserialize*

multimethod

Deserializes a primitive object into a Clojure data structure.

-

This is intended for use with types that are returned as a primitive but which need additional processing before they can be returned.

view source

deserialize-from

multimethod

Deserializes the given segment into a Clojure data structure.

+

This type of scope cannot be closed, and therefore should not be created in a with-open clause.

view source

copy-segment

(copy-segment dest src)

Copies the content to dest from src.

+

Returns dest.

view source

defalias

macro

(defalias new-type aliased-type)

Defines a type alias from new-type to aliased-type.

+

This creates needed serialization and deserialization implementations for the aliased type.

view source

deserialize

(deserialize obj type)

Deserializes an arbitrary type.

+

For types which have a primitive representation, this deserializes the primitive representation. For types which do not, this deserializes out of a segment.

view source

deserialize*

multimethod

Deserializes a primitive object into a Clojure data structure.

+

This is intended for use with types that are returned as a primitive but which need additional processing before they can be returned.

view source

deserialize-from

multimethod

Deserializes the given segment into a Clojure data structure.

For types that serialize to primitives, a default implementation will deserialize the primitive before calling deserialize*.

-

Implementations of this should be inside a with-acquired block for the the segment’s scope if they perform multiple memory operations.

view source

double-alignment

The alignment in bytes of a c-sized double.

view source

double-layout

The MemoryLayout for a c-sized double in native-endian ByteOrder.

view source

double-size

The size in bytes of a c-sized double.

view source

float-alignment

The alignment in bytes of a c-sized float.

view source

float-layout

The MemoryLayout for a c-sized float in native-endian ByteOrder.

view source

float-size

The size in bytes of a c-sized float.

view source

global-scope

(global-scope)

Constructs the global scope, which will never reclaim its resources.

-

This scope may be shared across threads, but is intended mainly in cases where memory is allocated with alloc but is either never freed or whose management is relinquished to a native library, such as when returned from a callback.

view source

int-alignment

The alignment in bytes of a c-sized int.

view source

int-layout

The MemoryLayout for a c-sized int in native-endian ByteOrder.

view source

int-size

The size in bytes of a c-sized int.

view source

java-layout

(java-layout type)

Gets the Java class to an argument of this type for a method handle.

-

If a type serializes to a primitive it returns return a Java primitive type. Otherwise, it returns MemorySegment.

view source

java-prim-layout

Map of primitive type names to the Java types for a method handle.

view source

little-endian

The little-endian ByteOrder.

-

See big-endian, native-endian

view source

long-alignment

The alignment in bytes of a c-sized long.

view source

long-layout

The MemoryLayout for a c-sized long in native-endian ByteOrder.

view source

long-size

The size in bytes of a c-sized long.

view source

native-endian

The ByteOrder for the native endianness of the current hardware.

-

See big-endian, little-endian.

view source

null?

(null? addr)

Checks if a memory address is null.

view source

pointer-alignment

The alignment in bytes of a c-sized pointer.

view source

pointer-layout

The MemoryLayout for a native pointer in native-endian ByteOrder.

view source

pointer-size

The size in bytes of a c-sized pointer.

view source

primitive-type

multimethod

Gets the primitive type that is used to pass as an argument for the type.

+

Implementations of this should be inside a with-acquired block for the the segment’s scope if they perform multiple memory operations.

view source

double-alignment

The alignment in bytes of a c-sized double.

view source

double-layout

The MemoryLayout for a c-sized double in native-endian ByteOrder.

view source

double-size

The size in bytes of a c-sized double.

view source

float-alignment

The alignment in bytes of a c-sized float.

view source

float-layout

The MemoryLayout for a c-sized float in native-endian ByteOrder.

view source

float-size

The size in bytes of a c-sized float.

view source

global-scope

(global-scope)

Constructs the global scope, which will never reclaim its resources.

+

This scope may be shared across threads, but is intended mainly in cases where memory is allocated with alloc but is either never freed or whose management is relinquished to a native library, such as when returned from a callback.

view source

int-alignment

The alignment in bytes of a c-sized int.

view source

int-layout

The MemoryLayout for a c-sized int in native-endian ByteOrder.

view source

int-size

The size in bytes of a c-sized int.

view source

java-layout

(java-layout type)

Gets the Java class to an argument of this type for a method handle.

+

If a type serializes to a primitive it returns return a Java primitive type. Otherwise, it returns MemorySegment.

view source

java-prim-layout

Map of primitive type names to the Java types for a method handle.

view source

little-endian

The little-endian ByteOrder.

+

See big-endian, native-endian

view source

long-alignment

The alignment in bytes of a c-sized long.

view source

long-layout

The MemoryLayout for a c-sized long in native-endian ByteOrder.

view source

long-size

The size in bytes of a c-sized long.

view source

native-endian

The ByteOrder for the native endianness of the current hardware.

+

See big-endian, little-endian.

view source

null?

(null? addr)

Checks if a memory address is null.

view source

pointer-alignment

The alignment in bytes of a c-sized pointer.

view source

pointer-layout

The MemoryLayout for a native pointer in native-endian ByteOrder.

view source

pointer-size

The size in bytes of a c-sized pointer.

view source

primitive-type

multimethod

Gets the primitive type that is used to pass as an argument for the type.

This is for objects which are passed to native functions as primitive types, but which need additional logic to be performed during serialization and deserialization.

Implementations of this method should take into account that type arguments may not always be evaluated before passing to this function.

-

Returns nil for any type which does not have a primitive representation.

view source

primitive-types

A set of all primitive types.

view source

primitive?

(primitive? type)

A predicate to determine if a given type is primitive.

view source

read-address

(read-address segment)(read-address segment offset)

Reads a MemoryAddress from the segment, at an optional offset.

view source

read-byte

(read-byte segment)(read-byte segment offset)

Reads a byte from the segment, at an optional offset.

view source

read-char

(read-char segment)(read-char segment offset)

Reads a char from the segment, at an optional offset.

view source

read-double

(read-double segment)(read-double segment offset)(read-double segment offset byte-order)

Reads a double from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

read-float

(read-float segment)(read-float segment offset)(read-float segment offset byte-order)

Reads a float from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

read-int

(read-int segment)(read-int segment offset)(read-int segment offset byte-order)

Reads a int from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

read-long

(read-long segment)(read-long segment offset)(read-long segment offset byte-order)

Reads a long from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

read-short

(read-short segment)(read-short segment offset)(read-short segment offset byte-order)

Reads a short from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

scope-allocator

(scope-allocator scope)

Constructs a segment allocator from the given scope.

-

This is primarily used when working with unwrapped downcall functions. When a downcall function returns a non-primitive type, it must be provided with an allocator.

view source

segment-scope

(segment-scope segment)

Gets the scope used to construct the segment.

view source

seq-of

(seq-of type segment)

Constructs a lazy sequence of type elements deserialized from segment.

view source

serialize

(serialize obj type)(serialize obj type scope)

Serializes an arbitrary type.

-

For types which have a primitive representation, this serializes into that representation. For types which do not, it allocates a new segment and serializes into that.

view source

serialize*

multimethod

Constructs a serialized version of the obj and returns it.

+

Returns nil for any type which does not have a primitive representation.

view source

primitive-types

A set of all primitive types.

view source

primitive?

(primitive? type)

A predicate to determine if a given type is primitive.

view source

read-address

(read-address segment)(read-address segment offset)

Reads a MemoryAddress from the segment, at an optional offset.

view source

read-byte

(read-byte segment)(read-byte segment offset)

Reads a byte from the segment, at an optional offset.

view source

read-char

(read-char segment)(read-char segment offset)

Reads a char from the segment, at an optional offset.

view source

read-double

(read-double segment)(read-double segment offset)(read-double segment offset byte-order)

Reads a double from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

read-float

(read-float segment)(read-float segment offset)(read-float segment offset byte-order)

Reads a float from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

read-int

(read-int segment)(read-int segment offset)(read-int segment offset byte-order)

Reads a int from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

read-long

(read-long segment)(read-long segment offset)(read-long segment offset byte-order)

Reads a long from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

read-short

(read-short segment)(read-short segment offset)(read-short segment offset byte-order)

Reads a short from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

scope-allocator

(scope-allocator scope)

Constructs a segment allocator from the given scope.

+

This is primarily used when working with unwrapped downcall functions. When a downcall function returns a non-primitive type, it must be provided with an allocator.

view source

segment-scope

(segment-scope segment)

Gets the scope used to construct the segment.

view source

seq-of

(seq-of type segment)

Constructs a lazy sequence of type elements deserialized from segment.

view source

serialize

(serialize obj type)(serialize obj type scope)

Serializes an arbitrary type.

+

For types which have a primitive representation, this serializes into that representation. For types which do not, it allocates a new segment and serializes into that.

view source

serialize*

multimethod

Constructs a serialized version of the obj and returns it.

Any new allocations made during the serialization should be tied to the given scope, except in extenuating circumstances.

-

This method should only be implemented for types that serialize to primitives.

view source

serialize-into

multimethod

Writes a serialized version of the obj to the given segment.

+

This method should only be implemented for types that serialize to primitives.

view source

serialize-into

multimethod

Writes a serialized version of the obj to the given segment.

Any new allocations made during the serialization should be tied to the given scope, except in extenuating circumstances.

This method should be implemented for any type which does not override c-layout.

For any other type, this will serialize it as serialize* before writing the result value into the segment.

-

Implementations of this should be inside a with-acquired block for the scope if they perform multiple memory operations.

view source

shared-scope

(shared-scope)

Constructs a new shared scope.

-

This scope can be shared across threads and memory allocated in it will only be cleaned up once every thread accessing the scope closes it.

view source

short-alignment

The alignment in bytes of a c-sized short.

view source

short-layout

The MemoryLayout for a c-sized short in native-endian ByteOrder.

view source

short-size

The size in bytes of a c-sized short.

view source

size-of

(size-of type)

The size in bytes of the given type.

view source

slice

(slice segment offset)(slice segment offset size)

Get a slice over the segment with the given offset.

view source

slice-into

(slice-into address segment)(slice-into address segment size)

Get a slice into the segment starting at the address.

view source

slice-segments

(slice-segments segment size)

Constructs a lazy seq of size-length memory segments, sliced from segment.

view source

stack-scope

(stack-scope)

Constructs a new scope for use only in this thread.

-

The memory allocated within this scope is cheap to allocate, like a native stack.

view source

with-acquired

macro

(with-acquired scopes & body)

Acquires one or more scopes until the body completes.

-

This is only necessary to do on shared scopes, however if you are operating on an arbitrary passed scope, it is best practice to wrap code that interacts with it wrapped in this.

view source

with-offset

(with-offset address offset)

Get a new address offset from the old address.

view source

write-address

(write-address segment value)(write-address segment offset value)

Writes a MemoryAddress to the segment, at an optional offset.

view source

write-byte

(write-byte segment value)(write-byte segment offset value)

Writes a byte to the segment, at an optional offset.

view source

write-char

(write-char segment value)(write-char segment offset value)

Writes a char to the segment, at an optional offset.

view source

write-double

(write-double segment value)(write-double segment offset value)(write-double segment offset byte-order value)

Writes a double to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

write-float

(write-float segment value)(write-float segment offset value)(write-float segment offset byte-order value)

Writes a float to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

write-int

(write-int segment value)(write-int segment offset value)(write-int segment offset byte-order value)

Writes a int to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

write-long

(write-long segment value)(write-long segment offset value)(write-long segment offset byte-order value)

Writes a long to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

write-short

(write-short segment value)(write-short segment offset value)(write-short segment offset byte-order value)

Writes a short to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source
\ No newline at end of file +

Implementations of this should be inside a with-acquired block for the scope if they perform multiple memory operations.

view source

shared-scope

(shared-scope)

Constructs a new shared scope.

+

This scope can be shared across threads and memory allocated in it will only be cleaned up once every thread accessing the scope closes it.

view source

short-alignment

The alignment in bytes of a c-sized short.

view source

short-layout

The MemoryLayout for a c-sized short in native-endian ByteOrder.

view source

short-size

The size in bytes of a c-sized short.

view source

size-of

(size-of type)

The size in bytes of the given type.

view source

slice

(slice segment offset)(slice segment offset size)

Get a slice over the segment with the given offset.

view source

slice-into

(slice-into address segment)(slice-into address segment size)

Get a slice into the segment starting at the address.

view source

slice-segments

(slice-segments segment size)

Constructs a lazy seq of size-length memory segments, sliced from segment.

view source

stack-scope

(stack-scope)

Constructs a new scope for use only in this thread.

+

The memory allocated within this scope is cheap to allocate, like a native stack.

view source

with-acquired

macro

(with-acquired scopes & body)

Acquires one or more scopes until the body completes.

+

This is only necessary to do on shared scopes, however if you are operating on an arbitrary passed scope, it is best practice to wrap code that interacts with it wrapped in this.

view source

with-offset

(with-offset address offset)

Get a new address offset from the old address.

view source

write-address

(write-address segment value)(write-address segment offset value)

Writes a MemoryAddress to the segment, at an optional offset.

view source

write-byte

(write-byte segment value)(write-byte segment offset value)

Writes a byte to the segment, at an optional offset.

view source

write-char

(write-char segment value)(write-char segment offset value)

Writes a char to the segment, at an optional offset.

view source

write-double

(write-double segment value)(write-double segment offset value)(write-double segment offset byte-order value)

Writes a double to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

write-float

(write-float segment value)(write-float segment offset value)(write-float segment offset byte-order value)

Writes a float to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

write-int

(write-int segment value)(write-int segment offset value)(write-int segment offset byte-order value)

Writes a int to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

write-long

(write-long segment value)(write-long segment offset value)(write-long segment offset byte-order value)

Writes a long to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

write-short

(write-short segment value)(write-short segment offset value)(write-short segment offset byte-order value)

Writes a short to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source
\ No newline at end of file diff --git a/docs/index.html b/docs/index.html index 3967959..ea9780b 100644 --- a/docs/index.html +++ b/docs/index.html @@ -1,3 +1,3 @@ -coffi v0.4.341

Project

Namespaces

coffi v0.4.341

A Foreign Function Interface in Clojure for JDK 17.

Namespaces

coffi.ffi

Functions for creating handles to native functions and loading native libraries.

Public variables and functions:

coffi.layout

Functions for adjusting the layout of structs.

Public variables and functions:

coffi.mem

Functions for managing native allocations, resource scopes, and (de)serialization.

Public variables and functions:

\ No newline at end of file +coffi v0.5.357

Project

Namespaces

coffi v0.5.357

A Foreign Function Interface in Clojure for JDK 18.

Namespaces

coffi.ffi

Functions for creating handles to native functions and loading native libraries.

Public variables and functions:

coffi.layout

Functions for adjusting the layout of structs.

Public variables and functions:

coffi.mem

Functions for managing native allocations, resource scopes, and (de)serialization.

Public variables and functions:

\ No newline at end of file From db242e1445e42f01f993ce305c711f8bb4607d64 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 11:03:26 -0500 Subject: [PATCH 05/50] Finish removing references to long-long --- src/clj/coffi/ffi.clj | 4 ---- 1 file changed, 4 deletions(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 6b4016f..999d25c 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -62,7 +62,6 @@ ::mem/short :sload ::mem/int :iload ::mem/long :lload - ::mem/long-long :lload ::mem/char :cload ::mem/float :fload ::mem/double :dload @@ -74,7 +73,6 @@ ::mem/short Short ::mem/int Integer ::mem/long Long - ::mem/long-long Long ::mem/char Character ::mem/float Float ::mem/double Double}) @@ -110,7 +108,6 @@ ::mem/short "shortValue" ::mem/int "intValue" ::mem/long "longValue" - ::mem/long-long "longValue" ::mem/char "charValue" ::mem/float "floatValue" ::mem/double "doubleValue"}) @@ -235,7 +232,6 @@ ::mem/short `short ::mem/int `int ::mem/long `long - ::mem/long-long `long ::mem/char `char ::mem/float `float ::mem/double `double}) From 98c59ea93645bb9ebedbd88913c2a6b77ea7af4d Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 11:03:51 -0500 Subject: [PATCH 06/50] Correct the docstring of `ensure-sybmol` --- src/clj/coffi/ffi.clj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 999d25c..0ea4c50 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -185,7 +185,7 @@ (insn/new-instance (downcall-class args ret) ^MethodHandle handle)) (defn- ensure-symbol - "Gets the address if the argument is [[Addressable]], otherwise + "Returns the argument if it is a [[NativeSymbol]], otherwise calls [[find-symbol]] on it." ^NativeSymbol [symbol-or-addr] (if (instance? NativeSymbol symbol-or-addr) From 01515ad56819b371e98c4f0e6dcd4985c53ec338 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 11:04:03 -0500 Subject: [PATCH 07/50] Correct the docstring of `static-variable` --- src/clj/coffi/ffi.clj | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 0ea4c50..6bcc5b9 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -600,8 +600,7 @@ (defn static-variable "Constructs a reference to a mutable value stored in `symbol-or-addr`. - The returned value can be dereferenced, and has metadata, and the address of - the value can be queried with [[address-of]]. + The returned value can be dereferenced, and has metadata. See [[freset!]], [[fswap!]]." [symbol-or-addr type] From b8383a467bdad0f8b10f79ac00520804e3836b3b Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 11:11:13 -0500 Subject: [PATCH 08/50] Correct docstring of downcall-handle --- src/clj/coffi/ffi.clj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 6bcc5b9..697e6c3 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -52,7 +52,7 @@ args-arr))))) (defn- downcall-handle - "Gets the [[MethodHandle]] for the function at the `address`." + "Gets the [[MethodHandle]] for the function at the `sym`." [sym function-descriptor] (.downcallHandle (CLinker/systemCLinker) sym function-descriptor)) From 3794ab7d5f0f50fdcdefaa81e6cec3ec743f2bce Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 11:11:21 -0500 Subject: [PATCH 09/50] Use coerce-addressable where possible --- src/clj/coffi/ffi.clj | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 697e6c3..5057346 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -506,10 +506,7 @@ ([args] (method-type args ::mem/void)) ([args ret] (MethodType/methodType - ^Class (let [r (mem/java-layout ret)] - (if (= r MemoryAddress) - Addressable - r)) + ^Class (coerce-addressable (mem/java-layout ret)) ^"[Ljava.lang.Class;" (into-array Class (map mem/java-layout args))))) (defn- upcall-handle From 20956f2549dc6cf11de04f3494af491ced65ec6d Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 13:05:39 -0500 Subject: [PATCH 10/50] Add function to get the segment from a static variable --- CHANGELOG.md | 2 ++ src/clj/coffi/ffi.clj | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 733d9e4..55813bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,8 @@ All notable changes to this project will be documented in this file. This change log follows the conventions of [keepachangelog.com](http://keepachangelog.com/). ## [Unreleased] +### Added +- New function to allow getting the backing memory segment of a `coffi.ffi.StaticVariable`, to replace the `Addressable` implementation lost in the migration to JDK 18 ## [0.5.357] - 2022-07-07 ### Removed diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 5057346..b68e8b0 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -19,6 +19,7 @@ FunctionDescriptor MemoryAddress MemoryLayout + MemorySegment NativeSymbol SegmentAllocator))) @@ -594,6 +595,14 @@ [static-var f & args] (freset! static-var (apply f @static-var args))) +(defn static-variable-segment + "Gets the backing [[MemorySegment]] from `static-var`. + + This is primarily useful when you need to pass the static variable's address + to a native function which takes an [[Addressable]]." + ^MemorySegment [static-var] + (.-seg ^StaticVariable static-var)) + (defn static-variable "Constructs a reference to a mutable value stored in `symbol-or-addr`. From d7e0ced38b64c88f875e2739278409dc886ccad0 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 13:27:02 -0500 Subject: [PATCH 11/50] Add macros to ease creating static variable references --- CHANGELOG.md | 1 + README.md | 13 ++++++++++--- src/clj/coffi/ffi.clj | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 55813bd..3042ed4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,7 @@ All notable changes to this project will be documented in this file. This change ## [Unreleased] ### Added +- New macros for defining vars with values from native code - New function to allow getting the backing memory segment of a `coffi.ffi.StaticVariable`, to replace the `Addressable` implementation lost in the migration to JDK 18 ## [0.5.357] - 2022-07-07 diff --git a/README.md b/README.md index c682235..409955f 100644 --- a/README.md +++ b/README.md @@ -265,18 +265,21 @@ does not support va-list, however it is a planned feature. ### Global Variables Some libraries include global variables or constants accessible through symbols. -To start with, constant values stored in symbols can be fetched with `const` +To start with, constant values stored in symbols can be fetched with `const`, or +the parallel macro `defconst` ```clojure (def some-const (ffi/const "some_const" ::mem/int)) +(ffi/defconst some-const "some_const" ::mem/int) ``` This value is fetched once when you call `const` and is turned into a Clojure -value. If you need to refer to a global variable, then `static-variable` can be -used to create a reference to the native value. +value. If you need to refer to a global variable, then `static-variable` (or +parallel `defvar`) can be used to create a reference to the native value. ```clojure (def some-var (ffi/static-variable "some_var" ::mem/int)) +(ffi/defvar some-var "some_var" ::mem/int) ``` This variable is an `IDeref`. Each time you dereference it, the value will be @@ -297,6 +300,10 @@ value is being mutated on another thread. A parallel function `fswap!` is also provided, but it does not provide any atomic semantics either. +The memory that backs the static variable can be fetched with the function +`static-variable-segment`, which can be used to pass a pointer to the static +variable to native functions that require it. + ### Complex Wrappers Some functions require more complex code to map nicely to a Clojure function. The `defcfn` macro provides facilities to wrap the native function with some diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index b68e8b0..41b76eb 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -561,6 +561,25 @@ [symbol-or-addr type] (mem/deserialize (.address (ensure-symbol symbol-or-addr)) [::mem/pointer type])) +(s/def ::defconst-args + (s/cat :var-name simple-symbol? + :docstring (s/? string?) + :symbol-or-addr any? + :type ::mem/type)) + +(defmacro defconst + "Defines a var named by `symbol` to be the value of the given `type` from `symbol-or-addr`." + {:arglists '([symbol docstring? symbol-or-addr type])} + [& args] + (let [args (s/conform ::defconst-args args)] + `(let [symbol# (ensure-symbol ~(:symbol-or-addr args))] + (def ~(:var-name args) + ~@(when-let [doc (:docstring args)] + (list doc)) + (const symbol# ~(:type args)))))) +(s/fdef defconst + :args ::defconst-args) + (deftype StaticVariable [seg type meta] IDeref (deref [_] @@ -615,6 +634,19 @@ (mem/global-scope)) type (atom nil))) +(defmacro defvar + "Defines a var named by `symbol` to be a reference to the native memory from `symbol-or-addr`." + {:arglists '([symbol docstring? symbol-or-addr type])} + [& args] + (let [args (s/conform ::defconst-args args)] + `(let [symbol# (ensure-symbol ~(:symbol-or-addr args))] + (def ~(:var-name args) + ~@(when-let [doc (:docstring args)] + (list doc)) + (static-variable symbol#))))) +(s/fdef defvar + :args ::defconst-args) + (s/def :coffi.ffi.symbolspec/symbol string?) (s/def :coffi.ffi.symbolspec/type keyword?) (s/def ::symbolspec From 2646e6fc9e7712bddffa88a225a7c423f640213b Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 14:15:33 -0500 Subject: [PATCH 12/50] Add enum serdes --- README.md | 2 +- src/clj/coffi/mem.clj | 43 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 409955f..589048b 100644 --- a/README.md +++ b/README.md @@ -1104,7 +1104,7 @@ These features are planned for future releases. - Support for va_args type - Header parsing tool for generating a data model? - Generic type aliases -- Helpers for generating enums & bitflags +- Helpers for generating bitflags - Unsigned integer types - Record-based struct types - Helper macro for out arguments diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 87fb28a..ce996cd 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -19,6 +19,7 @@ use [[with-acquired]] on the [[segment-scope]] in order to ensure it won't be released during its manipulation." (:require + [clojure.set :as set] [clojure.spec.alpha :as s]) (:import (java.nio ByteOrder) @@ -1235,6 +1236,48 @@ (slice-segments (slice segment 0 (* count (size-of type))) (size-of type)))) +;;; Enum types + +(defmethod primitive-type ::enum + [[_enum _variants & {:keys [repr]}]] + (if repr + (primitive-type repr) + ::int)) + +(defn- enum-variants-map + "Constructs a map from enum variant objects to their native representations. + + Enums are mappings from Clojure objects to numbers, with potential default + values for each element based on order. + + If `variants` is a map, then every variant has a value provided already (a + guarantee of maps in Clojure's syntax) and we are done. + + If `variants` is a vector then we assume C-style implicit enum values, + counting from 0. If an element of `variants` itself is a vector, it must be a + vector tuple of the variant object to the native representation, with further + counting continuing from that value." + [variants] + (if (map? variants) + variants + (reduce + (fn [[m next-id] variant] + (if (vector? variant) + [(conj m variant) (inc (second variant))] + [(assoc m variant next-id) (inc next-id)])) + [{} 0] + variants))) + +(defmethod serialize* ::enum + [obj [_enum variants & {:keys [repr]}] scope] + (serialize* ((enum-variants-map variants) obj) + (or repr ::int) + scope)) + +(defmethod deserialize* ::enum + [obj [_enum variants & {:keys [_repr]}]] + ((set/map-invert (enum-variants-map variants)) obj)) + (s/def ::type (s/spec (s/nonconforming From 6940842577d2c370840e5d4d5d10e7cd1ee5ac15 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 14:16:19 -0500 Subject: [PATCH 13/50] Add todo for custom serdes for composite types --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 589048b..71ae4ea 100644 --- a/README.md +++ b/README.md @@ -1110,6 +1110,7 @@ These features are planned for future releases. - Helper macro for out arguments - Improve error messages from defcfn macro - Mapped memory +- Helper macros for custom serde implementations for composite data types ### Future JDKs The purpose of coffi is to provide a wrapper for published versions of Project From c0dee46fe9c7054619871f2be42a59c19532a172 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 15:23:52 -0500 Subject: [PATCH 14/50] Fix bug with incorrect enum serdes --- src/clj/coffi/mem.clj | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index ce996cd..e95a518 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -1260,13 +1260,14 @@ [variants] (if (map? variants) variants - (reduce - (fn [[m next-id] variant] - (if (vector? variant) - [(conj m variant) (inc (second variant))] - [(assoc m variant next-id) (inc next-id)])) - [{} 0] - variants))) + (first + (reduce + (fn [[m next-id] variant] + (if (vector? variant) + [(conj m variant) (inc (second variant))] + [(assoc m variant next-id) (inc next-id)])) + [{} 0] + variants)))) (defmethod serialize* ::enum [obj [_enum variants & {:keys [repr]}] scope] From e9a5b640ac2498a8d45fe9ba8d21a188422d1c6d Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Thu, 7 Jul 2022 16:08:41 -0500 Subject: [PATCH 15/50] Add support for flagset composite types --- README.md | 1 - src/clj/coffi/mem.clj | 22 ++++++++++++++++++++++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 71ae4ea..461233e 100644 --- a/README.md +++ b/README.md @@ -1104,7 +1104,6 @@ These features are planned for future releases. - Support for va_args type - Header parsing tool for generating a data model? - Generic type aliases -- Helpers for generating bitflags - Unsigned integer types - Record-based struct types - Helper macro for out arguments diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index e95a518..792c04f 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -1279,6 +1279,28 @@ [obj [_enum variants & {:keys [_repr]}]] ((set/map-invert (enum-variants-map variants)) obj)) +;;; Flagsets + +(defmethod primitive-type ::flagset + [[_flagset _bits & {:keys [repr]}]] + (if repr + (primitive-type repr) + ::int)) + +(defmethod serialize* ::flagset + [obj [_flagset bits & {:keys [repr]}] scope] + (let [bits-map (enum-variants-map bits)] + (reduce #(bit-set %1 (get bits-map %2)) (serialize* 0 (or repr ::int) scope) obj))) + +(defmethod deserialize* ::flagset + [obj [_flagset bits & {:keys [repr]}]] + (let [bits-map (set/map-invert (enum-variants-map bits))] + (reduce #(if-not (zero? (bit-and 1 (bit-shift-right obj %2))) + (conj %1 (bits-map %2)) + %1) + #{} + (range (* 8 (size-of (or repr ::int))))))) + (s/def ::type (s/spec (s/nonconforming From c9f30a361ca9bd9d5ac1e2ddfeebbf2a076fedcd Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Sat, 16 Jul 2022 19:26:27 -0500 Subject: [PATCH 16/50] Add segment in readme about anonymous functions as callbacks --- README.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/README.md b/README.md index 461233e..d574274 100644 --- a/README.md +++ b/README.md @@ -239,6 +239,13 @@ type and message in the registers section, but it's important to be aware of all the same. Ideally you should test your callbacks before actually passing them to native code. +Another important thing to keep in mind is the expected lifetime of the function +that you pass to native code. For example it is perfectly fine to pass an +anonymous function to a native function if the callback will never be called +again once the native function returns. If however it saves the callback for +later use the JVM may collect it prematurely, causing a crash when the callback +is later called by native code. + ### Variadic Functions Some native functions can take any number of arguments, and in these cases coffi provides `vacfn-factory` (for "varargs C function factory"). From e1e64d836226ba642bdc050860d235910c829a59 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Sat, 16 Jul 2022 19:37:46 -0500 Subject: [PATCH 17/50] Add very inefficient raw types --- src/clj/coffi/mem.clj | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 792c04f..0cfa54b 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -1118,6 +1118,20 @@ (with-acquired [(segment-scope segment)] (map #(deserialize % type) (slice-segments segment (size-of type))))) +;;; Raw composite types + +(defmethod c-layout ::raw + [[_raw type]] + (c-layout type)) + +(defmethod serialize-into ::raw + [obj _type segment _scope] + (copy-segment segment obj)) + +(defmethod deserialize-from ::raw + [segment _type] + (clone-segment segment)) + ;;; C String type (defmethod primitive-type ::c-string From c0244a8b3383421fd53fd97e9de052485e66e74b Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Sat, 16 Jul 2022 19:46:45 -0500 Subject: [PATCH 18/50] Add todo about raw types --- src/clj/coffi/mem.clj | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 0cfa54b..c4642c1 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -1119,6 +1119,8 @@ (map #(deserialize % type) (slice-segments segment (size-of type))))) ;;; Raw composite types +;; TODO(Joshua): Ensure that all the raw values don't have anything happen on +;; serialize in the inlining of [[coffi.ffi/make-serde-wrapper]] (defmethod c-layout ::raw [[_raw type]] From 990e76c624a4a2b2d4f10b864c92c9c15b2500d5 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Mon, 31 Oct 2022 11:36:16 -0500 Subject: [PATCH 19/50] Fix incorrect pointer alignment --- CHANGELOG.md | 3 +++ src/clj/coffi/mem.clj | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3042ed4..2694a21 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,9 @@ All notable changes to this project will be documented in this file. This change - New macros for defining vars with values from native code - New function to allow getting the backing memory segment of a `coffi.ffi.StaticVariable`, to replace the `Addressable` implementation lost in the migration to JDK 18 +### Fixed +- Bug where pointer alignment was incorrectly defined + ## [0.5.357] - 2022-07-07 ### Removed - `:coffi.mem/long-long` primitive type diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index c4642c1..38135ad 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -236,7 +236,7 @@ (def ^ValueLayout$OfAddress pointer-layout "The [[MemoryLayout]] for a native pointer in [[native-endian]] [[ByteOrder]]." - ValueLayout/ADDRESS) + (MemoryLayout/valueLayout MemoryAddress native-endian)) (def ^long short-size "The size in bytes of a c-sized short." From 319bb3a33b81e64d2746d69dd5b4b058ba6be5a0 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Mon, 31 Oct 2022 11:36:38 -0500 Subject: [PATCH 20/50] Fix incorrect padding in C-layout structs --- CHANGELOG.md | 1 + src/clj/coffi/layout.clj | 7 ++++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2694a21..cfea49d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ All notable changes to this project will be documented in this file. This change - New function to allow getting the backing memory segment of a `coffi.ffi.StaticVariable`, to replace the `Addressable` implementation lost in the migration to JDK 18 ### Fixed +- Bug where padding in structs may be increased when fields have alignments less than their size - Bug where pointer alignment was incorrectly defined ## [0.5.357] - 2022-07-07 diff --git a/src/clj/coffi/layout.clj b/src/clj/coffi/layout.clj index 047bd62..15a2db6 100644 --- a/src/clj/coffi/layout.clj +++ b/src/clj/coffi/layout.clj @@ -16,11 +16,12 @@ (if (seq fields) (let [[[_ type :as field] & fields] fields size (mem/size-of type) - r (rem offset (mem/align-of type))] + align (mem/align-of type) + r (rem offset align)] (recur (cond-> (+ offset size) - (pos? r) (+ (- size r))) + (pos? r) (+ (- align r))) (cond-> aligned-fields - (pos? r) (conj [::padding [::mem/padding (- size r)]]) + (pos? r) (conj [::padding [::mem/padding (- align r)]]) :always (conj field)) fields)) (let [strongest-alignment (mem/align-of struct-spec) From 75bbe119717271f97b30f1d223b5dbadd60e0ed8 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Mon, 31 Oct 2022 11:39:56 -0500 Subject: [PATCH 21/50] Add todo about potential bug with nil as a primitive argument --- src/clj/coffi/ffi.clj | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 41b76eb..f673dea 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -247,6 +247,9 @@ [downcall arg-types ret-type] (let [;; Complexity of types const-args? (or (vector? arg-types) (nil? arg-types)) + ;; FIXME(Joshua): there's a potential bug where `nil` as a pointer + ;; argument is not converted to [[MemoryAddress/NULL]] if it's + ;; considered primitive. simple-args? (when const-args? (every? mem/primitive? arg-types)) const-ret? (s/valid? ::mem/type ret-type) From fa40902ce9671ca0711ce7852d7adcd88c7c20b3 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Mon, 31 Oct 2022 13:23:31 -0500 Subject: [PATCH 22/50] Fix bug with inlined serdes causing complex pointer serdes to fail --- CHANGELOG.md | 1 + src/clj/coffi/ffi.clj | 15 ++++++++++++--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cfea49d..e706cb0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ All notable changes to this project will be documented in this file. This change - New function to allow getting the backing memory segment of a `coffi.ffi.StaticVariable`, to replace the `Addressable` implementation lost in the migration to JDK 18 ### Fixed +- Bug where inline serde functions would fail on complex pointer types - Bug where padding in structs may be increased when fields have alignments less than their size - Bug where pointer alignment was incorrectly defined diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index f673dea..b95119b 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -251,10 +251,19 @@ ;; argument is not converted to [[MemoryAddress/NULL]] if it's ;; considered primitive. simple-args? (when const-args? - (every? mem/primitive? arg-types)) + (and (every? mem/primitive? arg-types) + ;; NOTE(Joshua): Pointer types with serdes (e.g. [::mem/pointer ::mem/int]) + ;; still require a scope, making them not qualify as "simple". + (every? keyword? (filter (comp #{::mem/pointer} mem/primitive-type) arg-types)))) const-ret? (s/valid? ::mem/type ret-type) - primitive-ret? (and const-ret? (or (mem/primitive? ret-type) - (#{::mem/void} ret-type))) + primitive-ret? (and const-ret? + (or (and (mem/primitive? ret-type) + ;; NOTE(Joshua): Pointer types with serdes require deserializing the + ;; return value, but don't require passing a scope to the downcall, + ;; making them cause the return to not be primitive, but it may still + ;; be "simple". + (or (keyword? ret-type) (not (#{::mem/pointer} (mem/primitive-type ret-type))))) + (#{::mem/void} ret-type))) simple-ret? (and const-ret? (mem/primitive-type ret-type)) no-serde? (and const-args? (empty? arg-types) primitive-ret?)] From 036d4112fbaae1f8f0837310e583760584f55e88 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Mon, 31 Oct 2022 13:38:07 -0500 Subject: [PATCH 23/50] Fix nullpointer serialization in simpler inline cases --- CHANGELOG.md | 1 + src/clj/coffi/ffi.clj | 6 ++---- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e706cb0..d91551b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ All notable changes to this project will be documented in this file. This change - New function to allow getting the backing memory segment of a `coffi.ffi.StaticVariable`, to replace the `Addressable` implementation lost in the migration to JDK 18 ### Fixed +- Bug where nil values would not be correctly coerced to null pointers when passed to inlined functions - Bug where inline serde functions would fail on complex pointer types - Bug where padding in structs may be increased when fields have alignments less than their size - Bug where pointer alignment was incorrectly defined diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index b95119b..d704993 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -247,9 +247,6 @@ [downcall arg-types ret-type] (let [;; Complexity of types const-args? (or (vector? arg-types) (nil? arg-types)) - ;; FIXME(Joshua): there's a potential bug where `nil` as a pointer - ;; argument is not converted to [[MemoryAddress/NULL]] if it's - ;; considered primitive. simple-args? (when const-args? (and (every? mem/primitive? arg-types) ;; NOTE(Joshua): Pointer types with serdes (e.g. [::mem/pointer ::mem/int]) @@ -300,8 +297,9 @@ (not (#{::mem/pointer} (mem/primitive-type type)))) (list (primitive-cast-sym (mem/primitive-type type)) sym) + ;; cast null pointers to something understood by panama (#{::mem/pointer} type) - nil + `(or ~sym (MemoryAddress/NULL)) (mem/primitive-type type) `(mem/serialize* ~sym ~type-sym ~scope) From 3838c0f13cd76ee9556da02e861354b2319a98ed Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 11 Nov 2022 13:53:52 -0600 Subject: [PATCH 24/50] Initial pass of update to JDK 19 --- build.clj | 6 +- deps.edn | 8 +- src/clj/coffi/ffi.clj | 21 +-- src/clj/coffi/mem.clj | 242 +++++++++++++++++---------------- src/java/coffi/ffi/Loader.java | 6 +- 5 files changed, 149 insertions(+), 134 deletions(-) diff --git a/build.clj b/build.clj index 21c254b..87d2cf0 100644 --- a/build.clj +++ b/build.clj @@ -49,11 +49,11 @@ "Compiles java classes required for interop." [opts] (.mkdirs (io/file class-dir)) - (b/process {:command-args ["javac" "--add-modules=jdk.incubator.foreign" + (b/process {:command-args ["javac" "--enable-preview" "src/java/coffi/ffi/Loader.java" "-d" class-dir - "-target" "18" - "-source" "18"]}) + "-target" "19" + "-source" "19"]}) opts) (defn- write-pom diff --git a/deps.edn b/deps.edn index 298cdeb..d410600 100644 --- a/deps.edn +++ b/deps.edn @@ -12,13 +12,13 @@ nodisassemble/nodisassemble {:mvn/version "0.1.3"}} ;; NOTE(Joshua): If you want to use nodisassemble you should also add a ;; -javaagent for the resolved location - :jvm-opts ["--add-modules=jdk.incubator.foreign" "--enable-native-access=ALL-UNNAMED"]} + :jvm-opts ["--enable-native-access=ALL-UNNAMED" "--enable-preview"]} :test {:extra-paths ["test/clj"] :extra-deps {org.clojure/test.check {:mvn/version "1.1.0"} io.github.cognitect-labs/test-runner {:git/url "https://github.com/cognitect-labs/test-runner" :sha "62ef1de18e076903374306060ac0e8a752e57c86"}} - :jvm-opts ["--add-modules=jdk.incubator.foreign" "--enable-native-access=ALL-UNNAMED"] + :jvm-opts ["--enable-native-access=ALL-UNNAMED" "--enable-preview"] :exec-fn cognitect.test-runner.api/test} :codox {:extra-deps {codox/codox {:mvn/version "0.10.7"}} @@ -30,8 +30,8 @@ :output-path "docs" :source-uri "https://github.com/IGJoshua/coffi/blob/{git-commit}/{filepath}#L{line}" :metadata {:doc/format :markdown}} - :jvm-opts ["--add-modules=jdk.incubator.foreign" - "--add-opens" "java.base/java.lang=ALL-UNNAMED"]} + :jvm-opts ["--add-opens" "java.base/java.lang=ALL-UNNAMED" + "--enable-preview"]} :build {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"} io.github.clojure/tools.build {:git/tag "v0.3.0" :git/sha "e418fc9"}} diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index d704993..2cc351a 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -13,16 +13,17 @@ MethodHandle MethodHandles MethodType) - (jdk.incubator.foreign + (java.lang.foreign Addressable - CLinker + Linker FunctionDescriptor MemoryAddress MemoryLayout MemorySegment - NativeSymbol SegmentAllocator))) +(set! *warn-on-reflection* true) + ;;; FFI Code loading and function access (defn load-system-library @@ -36,7 +37,7 @@ (Loader/loadLibrary (.getAbsolutePath (io/file path)))) (defn find-symbol - "Gets the [[NativeSymbol]] of a symbol from the loaded libraries." + "Gets the [[MemorySegment]] of a symbol from the loaded libraries." [sym] (Loader/findSymbol (name sym))) @@ -55,7 +56,7 @@ (defn- downcall-handle "Gets the [[MethodHandle]] for the function at the `sym`." [sym function-descriptor] - (.downcallHandle (CLinker/systemCLinker) sym function-descriptor)) + (.downcallHandle (Linker/nativeLinker) sym function-descriptor)) (def ^:private load-instructions "Mapping from primitive types to the instruction used to load them onto the stack." @@ -186,10 +187,10 @@ (insn/new-instance (downcall-class args ret) ^MethodHandle handle)) (defn- ensure-symbol - "Returns the argument if it is a [[NativeSymbol]], otherwise + "Returns the argument if it is a [[MemorySegment]], otherwise calls [[find-symbol]] on it." - ^NativeSymbol [symbol-or-addr] - (if (instance? NativeSymbol symbol-or-addr) + ^MemorySegment [symbol-or-addr] + (if (instance? MemorySegment symbol-or-addr) symbol-or-addr (find-symbol symbol-or-addr))) @@ -546,7 +547,7 @@ (defmethod mem/serialize* ::fn [f [_fn arg-types ret-type & {:keys [raw-fn?]}] scope] (.upcallStub - (CLinker/systemCLinker) + (Linker/nativeLinker) (cond-> f (not raw-fn?) (upcall-serde-wrapper arg-types ret-type) :always (upcall-handle arg-types ret-type)) @@ -558,7 +559,7 @@ (when-not (mem/null? addr) (vary-meta (-> addr - (as-> addr (NativeSymbol/ofAddress "coffi_upcall_symbol" addr (mem/connected-scope))) + (MemorySegment/ofAddress mem/pointer-size (mem/connected-scope)) (downcall-handle (function-descriptor arg-types ret-type)) (downcall-fn arg-types ret-type) (cond-> (not raw-fn?) (make-serde-wrapper arg-types ret-type))) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 38135ad..10d0bd8 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -22,32 +22,41 @@ [clojure.set :as set] [clojure.spec.alpha :as s]) (:import - (java.nio ByteOrder) - (jdk.incubator.foreign + (java.lang.foreign Addressable MemoryAddress MemoryLayout MemorySegment - ResourceScope + MemorySession SegmentAllocator ValueLayout - ValueLayout$OfAddress))) + ValueLayout$OfByte + ValueLayout$OfShort + ValueLayout$OfInt + ValueLayout$OfLong + ValueLayout$OfChar + ValueLayout$OfFloat + ValueLayout$OfDouble + ValueLayout$OfAddress) + (java.nio ByteOrder))) + +(set! *warn-on-reflection* true) (defn stack-scope "Constructs a new scope for use only in this thread. The memory allocated within this scope is cheap to allocate, like a native stack." - ^ResourceScope [] - (ResourceScope/newConfinedScope)) + ^MemorySession [] + (MemorySession/openConfined)) (defn shared-scope "Constructs a new shared scope. This scope can be shared across threads and memory allocated in it will only be cleaned up once every thread accessing the scope closes it." - ^ResourceScope [] - (ResourceScope/newSharedScope)) + ^MemorySession [] + (MemorySession/openShared)) (defn connected-scope "Constructs a new scope to reclaim all connected resources at once. @@ -57,8 +66,8 @@ This type of scope cannot be closed, and therefore should not be created in a [[with-open]] clause." - ^ResourceScope [] - (ResourceScope/newImplicitScope)) + ^MemorySession [] + (MemorySession/openImplicit)) (defn global-scope "Constructs the global scope, which will never reclaim its resources. @@ -67,8 +76,8 @@ memory is allocated with [[alloc]] but is either never freed or whose management is relinquished to a native library, such as when returned from a callback." - ^ResourceScope [] - (ResourceScope/globalScope)) + ^MemorySession [] + (MemorySession/global)) (defn scope-allocator "Constructs a segment allocator from the given `scope`. @@ -76,20 +85,21 @@ This is primarily used when working with unwrapped downcall functions. When a downcall function returns a non-primitive type, it must be provided with an allocator." - ^SegmentAllocator [^ResourceScope scope] - (SegmentAllocator/nativeAllocator scope)) + ^SegmentAllocator [^MemorySession scope] + (SegmentAllocator/newNativeArena scope)) (defn segment-scope "Gets the scope used to construct the `segment`." - ^ResourceScope [segment] - (.scope ^MemorySegment segment)) + ^MemorySession [segment] + (.session ^MemorySegment segment)) (defn alloc "Allocates `size` bytes. If a `scope` is provided, the allocation will be reclaimed when it is closed." (^MemorySegment [size] (alloc size (connected-scope))) - (^MemorySegment [size scope] (MemorySegment/allocateNative (long size) ^ResourceScope scope))) + (^MemorySegment [size scope] (MemorySegment/allocateNative (long size) ^MemorySession scope)) + (^MemorySegment [size alignment scope] (MemorySegment/allocateNative (long size) (long alignment) ^MemorySession scope))) (defn alloc-with "Allocates `size` bytes using the `allocator`." @@ -106,10 +116,14 @@ with it wrapped in this." {:style/indent 1} [scopes & body] - `(with-open [scope# (stack-scope)] - (doseq [target-scope# (vec ~scopes)] - (.keepAlive scope# target-scope#)) - ~@body)) + (if (seq scopes) + `(let [scope# ~(first scopes)] + (.whileAlive + ^MemorySession scope# + (^:once fn* [] + (with-acquired [~@(rest scopes)] + ~@body)))) + `(do ~@body))) (s/fdef with-acquired :args (s/cat :scopes any? :body (s/* any?))) @@ -154,7 +168,7 @@ (defn add-close-action! "Adds a 0-arity function to be run when the `scope` closes." - [^ResourceScope scope ^Runnable action] + [^MemorySession scope ^Runnable action] (.addCloseAction scope action) nil) @@ -206,37 +220,37 @@ See [[big-endian]], [[little-endian]]." (ByteOrder/nativeOrder)) -(def ^ValueLayout byte-layout +(def ^ValueLayout$OfByte byte-layout "The [[MemoryLayout]] for a byte in [[native-endian]] [[ByteOrder]]." - (MemoryLayout/valueLayout Byte/TYPE native-endian)) + ValueLayout/JAVA_BYTE) -(def ^ValueLayout short-layout +(def ^ValueLayout$OfShort short-layout "The [[MemoryLayout]] for a c-sized short in [[native-endian]] [[ByteOrder]]." - (MemoryLayout/valueLayout Short/TYPE native-endian)) + ValueLayout/JAVA_SHORT) -(def ^ValueLayout int-layout +(def ^ValueLayout$OfInt int-layout "The [[MemoryLayout]] for a c-sized int in [[native-endian]] [[ByteOrder]]." - (MemoryLayout/valueLayout Integer/TYPE native-endian)) + ValueLayout/JAVA_INT) -(def ^ValueLayout long-layout +(def ^ValueLayout$OfLong long-layout "The [[MemoryLayout]] for a c-sized long in [[native-endian]] [[ByteOrder]]." - (MemoryLayout/valueLayout Long/TYPE native-endian)) + ValueLayout/JAVA_LONG) -(def ^ValueLayout char-layout +(def ^ValueLayout$OfByte char-layout "The [[MemoryLayout]] for a c-sized char in [[native-endian]] [[ByteOrder]]." - (MemoryLayout/valueLayout Byte/TYPE native-endian)) + ValueLayout/JAVA_BYTE) -(def ^ValueLayout float-layout +(def ^ValueLayout$OfFloat float-layout "The [[MemoryLayout]] for a c-sized float in [[native-endian]] [[ByteOrder]]." - (MemoryLayout/valueLayout Float/TYPE native-endian)) + ValueLayout/JAVA_FLOAT) -(def ^ValueLayout double-layout +(def ^ValueLayout$OfDouble double-layout "The [[MemoryLayout]] for a c-sized double in [[native-endian]] [[ByteOrder]]." - (MemoryLayout/valueLayout Double/TYPE native-endian)) + ValueLayout/JAVA_DOUBLE) (def ^ValueLayout$OfAddress pointer-layout "The [[MemoryLayout]] for a native pointer in [[native-endian]] [[ByteOrder]]." - (MemoryLayout/valueLayout MemoryAddress native-endian)) + ValueLayout/ADDRESS) (def ^long short-size "The size in bytes of a c-sized short." @@ -292,15 +306,15 @@ (fn read-byte-inline ([segment] `(let [segment# ~segment] - (.get ^MemorySegment segment# ^ValueLayout byte-layout 0))) + (.get ^MemorySegment segment# ^ValueLayout$OfByte byte-layout 0))) ([segment offset] `(let [segment# ~segment offset# ~offset] - (.get ^MemorySegment segment# ^ValueLayout byte-layout offset#))))} + (.get ^MemorySegment segment# ^ValueLayout$OfByte byte-layout offset#))))} ([^MemorySegment segment] - (.get segment ^ValueLayout byte-layout 0)) + (.get segment ^ValueLayout$OfByte byte-layout 0)) ([^MemorySegment segment ^long offset] - (.get segment ^ValueLayout byte-layout offset))) + (.get segment ^ValueLayout$OfByte byte-layout offset))) (defn read-short "Reads a [[short]] from the `segment`, at an optional `offset`. @@ -310,22 +324,22 @@ (fn read-short-inline ([segment] `(let [segment# ~segment] - (.get ^MemorySegment segment# ^ValueLayout short-layout 0))) + (.get ^MemorySegment segment# ^ValueLayout$OfShort short-layout 0))) ([segment offset] `(let [segment# ~segment offset# ~offset] - (.get ^MemorySegment segment# ^ValueLayout short-layout offset#))) + (.get ^MemorySegment segment# ^ValueLayout$OfShort short-layout offset#))) ([segment offset byte-order] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order] - (.get ^MemorySegment segment# (.withOrder ^ValueLayout short-layout ^ByteOrder byte-order#) offset#))))} + (.get ^MemorySegment segment# (.withOrder ^ValueLayout$OfShort short-layout ^ByteOrder byte-order#) offset#))))} ([^MemorySegment segment] - (.get segment ^ValueLayout short-layout 0)) + (.get segment ^ValueLayout$OfShort short-layout 0)) ([^MemorySegment segment ^long offset] - (.get segment ^ValueLayout short-layout offset)) + (.get segment ^ValueLayout$OfShort short-layout offset)) ([^MemorySegment segment ^long offset ^ByteOrder byte-order] - (.get segment (.withOrder ^ValueLayout short-layout byte-order) offset))) + (.get segment (.withOrder ^ValueLayout$OfShort short-layout byte-order) offset))) (defn read-int "Reads a [[int]] from the `segment`, at an optional `offset`. @@ -335,22 +349,22 @@ (fn read-int-inline ([segment] `(let [segment# ~segment] - (.get ^MemorySegment segment# ^ValueLayout int-layout 0))) + (.get ^MemorySegment segment# ^ValueLayout$OfInt int-layout 0))) ([segment offset] `(let [segment# ~segment offset# ~offset] - (.get ^MemorySegment segment# ^ValueLayout int-layout offset#))) + (.get ^MemorySegment segment# ^ValueLayout$OfInt int-layout offset#))) ([segment offset byte-order] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order] - (.get ^MemorySegment segment# (.withOrder ^ValueLayout int-layout ^ByteOrder byte-order#) offset#))))} + (.get ^MemorySegment segment# (.withOrder ^ValueLayout$OfInt int-layout ^ByteOrder byte-order#) offset#))))} ([^MemorySegment segment] - (.get segment ^ValueLayout int-layout 0)) + (.get segment ^ValueLayout$OfInt int-layout 0)) ([^MemorySegment segment ^long offset] - (.get segment ^ValueLayout int-layout offset)) + (.get segment ^ValueLayout$OfInt int-layout offset)) ([^MemorySegment segment ^long offset ^ByteOrder byte-order] - (.get segment (.withOrder ^ValueLayout int-layout byte-order) offset))) + (.get segment (.withOrder ^ValueLayout$OfInt int-layout byte-order) offset))) (defn read-long "Reads a [[long]] from the `segment`, at an optional `offset`. @@ -360,22 +374,22 @@ (fn read-long-inline ([segment] `(let [segment# ~segment] - (.get ^MemorySegment segment# ^ValueLayout long-layout 0))) + (.get ^MemorySegment segment# ^ValueLayout$OfLong long-layout 0))) ([segment offset] `(let [segment# ~segment offset# ~offset] - (.get ^MemorySegment segment# ^ValueLayout long-layout offset#))) + (.get ^MemorySegment segment# ^ValueLayout$OfLong long-layout offset#))) ([segment offset byte-order] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order] - (.get ^MemorySegment segment# (.withOrder ^ValueLayout long-layout ^ByteOrder byte-order#) offset#))))} + (.get ^MemorySegment segment# (.withOrder ^ValueLayout$OfLong long-layout ^ByteOrder byte-order#) offset#))))} (^long [^MemorySegment segment] - (.get segment ^ValueLayout long-layout 0)) + (.get segment ^ValueLayout$OfLong long-layout 0)) (^long [^MemorySegment segment ^long offset] - (.get segment ^ValueLayout long-layout offset)) + (.get segment ^ValueLayout$OfLong long-layout offset)) (^long [^MemorySegment segment ^long offset ^ByteOrder byte-order] - (.get segment (.withOrder ^ValueLayout long-layout byte-order) offset))) + (.get segment (.withOrder ^ValueLayout$OfLong long-layout byte-order) offset))) (defn read-char "Reads a [[char]] from the `segment`, at an optional `offset`." @@ -383,15 +397,15 @@ (fn read-char-inline ([segment] `(let [segment# ~segment] - (char (Byte/toUnsignedInt (.get ^MemorySegment segment# ^ValueLayout byte-layout 0))))) + (char (Byte/toUnsignedInt (.get ^MemorySegment segment# ^ValueLayout$OfByte byte-layout 0))))) ([segment offset] `(let [segment# ~segment offset# ~offset] - (char (Byte/toUnsignedInt (.get ^MemorySegment segment# ^ValueLayout byte-layout offset#))))))} + (char (Byte/toUnsignedInt (.get ^MemorySegment segment# ^ValueLayout$OfByte byte-layout offset#))))))} ([^MemorySegment segment] - (char (Byte/toUnsignedInt (.get segment ^ValueLayout byte-layout 0)))) + (char (Byte/toUnsignedInt (.get segment ^ValueLayout$OfChar byte-layout 0)))) ([^MemorySegment segment ^long offset] - (char (Byte/toUnsignedInt (.get segment ^ValueLayout byte-layout offset))))) + (char (Byte/toUnsignedInt (.get segment ^ValueLayout$OfChar byte-layout offset))))) (defn read-float "Reads a [[float]] from the `segment`, at an optional `offset`. @@ -401,22 +415,22 @@ (fn read-float-inline ([segment] `(let [segment# ~segment] - (.get ^MemorySegment segment# ^ValueLayout float-layout 0))) + (.get ^MemorySegment segment# ^ValueLayout$OfFloat float-layout 0))) ([segment offset] `(let [segment# ~segment offset# ~offset] - (.get ^MemorySegment segment# ^ValueLayout float-layout offset#))) + (.get ^MemorySegment segment# ^ValueLayout$OfFloat float-layout offset#))) ([segment offset byte-order] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order] - (.get ^MemorySegment segment# (.withOrder ^ValueLayout float-layout ^ByteOrder byte-order#) offset#))))} + (.get ^MemorySegment segment# (.withOrder ^ValueLayout$OfFloat float-layout ^ByteOrder byte-order#) offset#))))} ([^MemorySegment segment] - (.get segment ^ValueLayout float-layout 0)) + (.get segment ^ValueLayout$OfFloat float-layout 0)) ([^MemorySegment segment ^long offset] - (.get segment ^ValueLayout float-layout offset)) + (.get segment ^ValueLayout$OfFloat float-layout offset)) ([^MemorySegment segment ^long offset ^ByteOrder byte-order] - (.get segment (.withOrder ^ValueLayout float-layout byte-order) offset))) + (.get segment (.withOrder ^ValueLayout$OfFloat float-layout byte-order) offset))) (defn read-double "Reads a [[double]] from the `segment`, at an optional `offset`. @@ -426,22 +440,22 @@ (fn read-double-inline ([segment] `(let [segment# ~segment] - (.get ^MemorySegment segment# ^ValueLayout double-layout 0))) + (.get ^MemorySegment segment# ^ValueLayout$OfDouble double-layout 0))) ([segment offset] `(let [segment# ~segment offset# ~offset] - (.get ^MemorySegment segment# ^ValueLayout double-layout offset#))) + (.get ^MemorySegment segment# ^ValueLayout$OfDouble double-layout offset#))) ([segment offset byte-order] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order] - (.get ^MemorySegment segment# (.withOrder ^ValueLayout double-layout ^ByteOrder byte-order#) offset#))))} + (.get ^MemorySegment segment# (.withOrder ^ValueLayout$OfDouble double-layout ^ByteOrder byte-order#) offset#))))} (^double [^MemorySegment segment] - (.get segment ^ValueLayout double-layout 0)) + (.get segment ^ValueLayout$OfDouble double-layout 0)) (^double [^MemorySegment segment ^long offset] - (.get segment ^ValueLayout double-layout offset)) + (.get segment ^ValueLayout$OfDouble double-layout offset)) (^double [^MemorySegment segment ^long offset ^ByteOrder byte-order] - (.get segment (.withOrder ^ValueLayout double-layout byte-order) offset))) + (.get segment (.withOrder ^ValueLayout$OfDouble double-layout byte-order) offset))) (defn read-address "Reads a [[MemoryAddress]] from the `segment`, at an optional `offset`." @@ -466,16 +480,16 @@ ([segment value] `(let [segment# ~segment value# ~value] - (.set ^MemorySegment segment# ^ValueLayout byte-layout 0 value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfByte byte-layout 0 value#))) ([segment offset value] `(let [segment# ~segment offset# ~offset value# ~value] - (.set ^MemorySegment segment# ^ValueLayout byte-layout offset# value#))))} + (.set ^MemorySegment segment# ^ValueLayout$OfByte byte-layout offset# value#))))} ([^MemorySegment segment value] - (.set segment ^ValueLayout byte-layout 0 ^byte value)) + (.set segment ^ValueLayout$OfByte byte-layout 0 ^byte value)) ([^MemorySegment segment ^long offset value] - (.set segment ^ValueLayout byte-layout offset ^byte value))) + (.set segment ^ValueLayout$OfByte byte-layout offset ^byte value))) (defn write-short "Writes a [[short]] to the `segment`, at an optional `offset`. @@ -486,24 +500,24 @@ ([segment value] `(let [segment# ~segment value# ~value] - (.set ^MemorySegment segment# ^ValueLayout short-layout 0 value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfShort short-layout 0 value#))) ([segment offset value] `(let [segment# ~segment offset# ~offset value# ~value] - (.set ^MemorySegment segment# ^ValueLayout short-layout offset# value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfShort short-layout offset# value#))) ([segment offset byte-order value] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order value# ~value] - (.set ^MemorySegment segment# (.withOrder ^ValueLayout short-layout ^ByteOrder byte-order#) offset# value#))))} + (.set ^MemorySegment segment# (.withOrder ^ValueLayout$OfShort short-layout ^ByteOrder byte-order#) offset# value#))))} ([^MemorySegment segment value] - (.set segment ^ValueLayout short-layout 0 ^short value)) + (.set segment ^ValueLayout$OfShort short-layout 0 ^short value)) ([^MemorySegment segment ^long offset value] - (.set segment ^ValueLayout short-layout offset ^short value)) + (.set segment ^ValueLayout$OfShort short-layout offset ^short value)) ([^MemorySegment segment ^long offset ^ByteOrder byte-order value] - (.set segment (.withOrder ^ValueLayout short-layout byte-order) offset ^short value))) + (.set segment (.withOrder ^ValueLayout$OfShort short-layout byte-order) offset ^short value))) (defn write-int "Writes a [[int]] to the `segment`, at an optional `offset`. @@ -514,24 +528,24 @@ ([segment value] `(let [segment# ~segment value# ~value] - (.set ^MemorySegment segment# ^ValueLayout int-layout 0 value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfInt int-layout 0 value#))) ([segment offset value] `(let [segment# ~segment offset# ~offset value# ~value] - (.set ^MemorySegment segment# ^ValueLayout int-layout offset# value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfInt int-layout offset# value#))) ([segment offset byte-order value] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order value# ~value] - (.set ^MemorySegment segment# (.withOrder ^ValueLayout int-layout ^ByteOrder byte-order#) offset# value#))))} + (.set ^MemorySegment segment# (.withOrder ^ValueLayout$OfInt int-layout ^ByteOrder byte-order#) offset# value#))))} ([^MemorySegment segment value] - (.set segment ^ValueLayout int-layout 0 ^int value)) + (.set segment ^ValueLayout$OfInt int-layout 0 ^int value)) ([^MemorySegment segment ^long offset value] - (.set segment ^ValueLayout int-layout offset ^int value)) + (.set segment ^ValueLayout$OfInt int-layout offset ^int value)) ([^MemorySegment segment ^long offset ^ByteOrder byte-order value] - (.set segment (.withOrder ^ValueLayout int-layout byte-order) offset ^int value))) + (.set segment (.withOrder ^ValueLayout$OfInt int-layout byte-order) offset ^int value))) (defn write-long "Writes a [[long]] to the `segment`, at an optional `offset`. @@ -542,24 +556,24 @@ ([segment value] `(let [segment# ~segment value# ~value] - (.set ^MemorySegment segment# ^ValueLayout long-layout 0 value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfLong long-layout 0 value#))) ([segment offset value] `(let [segment# ~segment offset# ~offset value# ~value] - (.set ^MemorySegment segment# ^ValueLayout long-layout offset# value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfLong long-layout offset# value#))) ([segment offset byte-order value] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order value# ~value] - (.set ^MemorySegment segment# (.withOrder ^ValueLayout long-layout ^ByteOrder byte-order#) offset# value#))))} + (.set ^MemorySegment segment# (.withOrder ^ValueLayout$OfLong long-layout ^ByteOrder byte-order#) offset# value#))))} (^long [^MemorySegment segment ^long value] - (.set segment ^ValueLayout long-layout 0 value)) + (.set segment ^ValueLayout$OfLong long-layout 0 value)) (^long [^MemorySegment segment ^long offset ^long value] - (.set segment ^ValueLayout long-layout offset value)) + (.set segment ^ValueLayout$OfLong long-layout offset value)) (^long [^MemorySegment segment ^long offset ^ByteOrder byte-order ^long value] - (.set segment (.withOrder ^ValueLayout long-layout byte-order) offset value))) + (.set segment (.withOrder ^ValueLayout$OfLong long-layout byte-order) offset value))) (defn write-char "Writes a [[char]] to the `segment`, at an optional `offset`." @@ -568,22 +582,22 @@ ([segment value] `(let [segment# ~segment value# ~value] - (.set ^MemorySegment segment# ^ValueLayout byte-layout 0 (unchecked-byte (unchecked-int value#))))) + (.set ^MemorySegment segment# ^ValueLayout$OfByte byte-layout 0 (unchecked-byte (unchecked-int value#))))) ([segment offset value] `(let [segment# ~segment offset# ~offset value# ~value] - (.set ^MemorySegment segment# ^ValueLayout byte-layout offset# (unchecked-byte (unchecked-int value#))))))} + (.set ^MemorySegment segment# ^ValueLayout$OfByte byte-layout offset# (unchecked-byte (unchecked-int value#))))))} ([^MemorySegment segment value] (.set segment ;; HACK(Joshua): The Clojure runtime doesn't have an unchecked-byte cast for ;; characters, so this double cast is necessary unless I emit ;; my own bytecode with insn. - ^ValueLayout byte-layout 0 + ^ValueLayout$OfByte byte-layout 0 (unchecked-byte (unchecked-int ^char value)))) ([^MemorySegment segment ^long offset value] - (.set segment ^ValueLayout byte-layout offset (unchecked-byte (unchecked-int ^char value))))) + (.set segment ^ValueLayout$OfByte byte-layout offset (unchecked-byte (unchecked-int ^char value))))) (defn write-float "Writes a [[float]] to the `segment`, at an optional `offset`. @@ -594,24 +608,24 @@ ([segment value] `(let [segment# ~segment value# ~value] - (.set ^MemorySegment segment# ^ValueLayout float-layout 0 value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfFloat float-layout 0 value#))) ([segment offset value] `(let [segment# ~segment offset# ~offset value# ~value] - (.set ^MemorySegment segment# ^ValueLayout float-layout offset# value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfFloat float-layout offset# value#))) ([segment offset byte-order value] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order value# ~value] - (.set ^MemorySegment segment# (.withOrder ^ValueLayout float-layout ^ByteOrder byte-order#) offset# value#))))} + (.set ^MemorySegment segment# (.withOrder ^ValueLayout$OfFloat float-layout ^ByteOrder byte-order#) offset# value#))))} ([^MemorySegment segment value] - (.set segment ^ValueLayout float-layout 0 ^float value)) + (.set segment ^ValueLayout$OfFloat float-layout 0 ^float value)) ([^MemorySegment segment ^long offset value] - (.set segment ^ValueLayout float-layout offset ^float value)) + (.set segment ^ValueLayout$OfFloat float-layout offset ^float value)) ([^MemorySegment segment ^long offset ^ByteOrder byte-order value] - (.set segment (.withOrder ^ValueLayout float-layout byte-order) offset ^float value))) + (.set segment (.withOrder ^ValueLayout$OfFloat float-layout byte-order) offset ^float value))) (defn write-double "Writes a [[double]] to the `segment`, at an optional `offset`. @@ -622,24 +636,24 @@ ([segment value] `(let [segment# ~segment value# ~value] - (.set ^MemorySegment segment# ^ValueLayout double-layout 0 value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfDouble double-layout 0 value#))) ([segment offset value] `(let [segment# ~segment offset# ~offset value# ~value] - (.set ^MemorySegment segment# ^ValueLayout double-layout offset# value#))) + (.set ^MemorySegment segment# ^ValueLayout$OfDouble double-layout offset# value#))) ([segment offset byte-order value] `(let [segment# ~segment offset# ~offset byte-order# ~byte-order value# ~value] - (.set ^MemorySegment segment# (.withOrder ^ValueLayout double-layout ^ByteOrder byte-order#) offset# value#))))} + (.set ^MemorySegment segment# (.withOrder ^ValueLayout$OfDouble double-layout ^ByteOrder byte-order#) offset# value#))))} (^double [^MemorySegment segment ^double value] - (.set segment ^ValueLayout double-layout 0 value)) + (.set segment ^ValueLayout$OfDouble double-layout 0 value)) (^double [^MemorySegment segment ^long offset ^double value] - (.set segment ^ValueLayout double-layout offset value)) + (.set segment ^ValueLayout$OfDouble double-layout offset value)) (^double [^MemorySegment segment ^long offset ^ByteOrder byte-order ^double value] - (.set segment (.withOrder ^ValueLayout double-layout byte-order) offset value))) + (.set segment (.withOrder ^ValueLayout$OfDouble double-layout byte-order) offset value))) (defn write-address "Writes a [[MemoryAddress]] to the `segment`, at an optional `offset`." @@ -822,7 +836,7 @@ (defn alloc-instance "Allocates a memory segment for the given `type`." (^MemorySegment [type] (alloc-instance type (connected-scope))) - (^MemorySegment [type scope] (MemorySegment/allocateNative ^long (size-of type) ^ResourceScope scope))) + (^MemorySegment [type scope] (MemorySegment/allocateNative ^long (size-of type) ^MemorySession scope))) (declare serialize serialize-into) diff --git a/src/java/coffi/ffi/Loader.java b/src/java/coffi/ffi/Loader.java index 9ad662f..8830118 100644 --- a/src/java/coffi/ffi/Loader.java +++ b/src/java/coffi/ffi/Loader.java @@ -1,6 +1,6 @@ package coffi.ffi; -import jdk.incubator.foreign.*; +import java.lang.foreign.*; /** * Loading libraries with the {@link System#load} and {@link System#loadLibrary} @@ -36,8 +36,8 @@ public class Loader { * * @param symbol The name of the symbol to load from a library. */ - public static NativeSymbol findSymbol(String symbol) { - return CLinker.systemCLinker().lookup(symbol) + public static MemorySegment findSymbol(String symbol) { + return Linker.nativeLinker().defaultLookup().lookup(symbol) .orElseGet(() -> SymbolLookup.loaderLookup().lookup(symbol).orElse(null)); } } From 49bdca07660187e9e335e5192e235430dd3cca62 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 11 Nov 2022 14:07:38 -0600 Subject: [PATCH 25/50] Rename scope to session everywhere --- src/clj/coffi/ffi.clj | 52 ++++----- src/clj/coffi/mem.clj | 249 +++++++++++++++++++++++++----------------- 2 files changed, 176 insertions(+), 125 deletions(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 2cc351a..a455fe9 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -244,20 +244,20 @@ The return type and any arguments that are primitives will not be (de)serialized except to be cast. If all arguments and return are primitive, the `downcall` is returned directly. In cases where arguments must - be serialized, a new [[mem/stack-scope]] is generated." + be serialized, a new [[mem/stack-session]] is generated." [downcall arg-types ret-type] (let [;; Complexity of types const-args? (or (vector? arg-types) (nil? arg-types)) simple-args? (when const-args? (and (every? mem/primitive? arg-types) ;; NOTE(Joshua): Pointer types with serdes (e.g. [::mem/pointer ::mem/int]) - ;; still require a scope, making them not qualify as "simple". + ;; still require a session, making them not qualify as "simple". (every? keyword? (filter (comp #{::mem/pointer} mem/primitive-type) arg-types)))) const-ret? (s/valid? ::mem/type ret-type) primitive-ret? (and const-ret? (or (and (mem/primitive? ret-type) ;; NOTE(Joshua): Pointer types with serdes require deserializing the - ;; return value, but don't require passing a scope to the downcall, + ;; return value, but don't require passing a session to the downcall, ;; making them cause the return to not be primitive, but it may still ;; be "simple". (or (keyword? ret-type) (not (#{::mem/pointer} (mem/primitive-type ret-type))))) @@ -273,7 +273,7 @@ ~ret-type downcall#) (let [;; All our symbols - scope (gensym "scope") + session (gensym "session") downcall-sym (gensym "downcall") args-sym (when-not const-args? (gensym "args")) @@ -292,7 +292,7 @@ (some->> (cond (not (s/valid? ::mem/type type)) - `(mem/serialize ~sym ~type-sym ~scope) + `(mem/serialize ~sym ~type-sym ~session) (and (mem/primitive? type) (not (#{::mem/pointer} (mem/primitive-type type)))) @@ -303,11 +303,11 @@ `(or ~sym (MemoryAddress/NULL)) (mem/primitive-type type) - `(mem/serialize* ~sym ~type-sym ~scope) + `(mem/serialize* ~sym ~type-sym ~session) :else `(let [alloc# (mem/alloc-instance ~type-sym)] - (mem/serialize-into ~sym ~type-sym alloc# ~scope) + (mem/serialize-into ~sym ~type-sym alloc# ~session) alloc#)) (list sym))) @@ -334,7 +334,7 @@ :else `(let [~args-sym (map (fn [obj# type#] - (mem/serialize obj# type# ~scope)) + (mem/serialize obj# type# ~session)) ~args-sym ~args-types-sym)] ~expr))) @@ -343,7 +343,7 @@ ;; taking restargs, and so the downcall must be applied (-> `(~@(when (symbol? args) [`apply]) ~downcall-sym - ~@(when allocator? [`(mem/scope-allocator ~scope)]) + ~@(when allocator? [`(mem/session-allocator ~session)]) ~@(if (symbol? args) [args] args)) @@ -366,12 +366,12 @@ :else (deserialize-segment expr))) - wrap-scope (fn [expr] - `(with-open [~scope (mem/stack-scope)] - ~expr)) - wrap-fn (fn [call needs-scope?] + wrap-session (fn [expr] + `(with-open [~session (mem/stack-session)] + ~expr)) + wrap-fn (fn [call needs-session?] `(fn [~@(if const-args? arg-syms ['& args-sym])] - ~(cond-> call needs-scope? wrap-scope)))] + ~(cond-> call needs-session? wrap-session)))] `(let [;; NOTE(Joshua): To ensure all arguments are evaluated once and ;; in-order, they must be bound here ~downcall-sym ~downcall @@ -403,15 +403,15 @@ [downcall arg-types ret-type] (if (mem/primitive-type ret-type) (fn native-fn [& args] - (with-open [scope (mem/stack-scope)] + (with-open [session (mem/stack-session)] (mem/deserialize* - (apply downcall (map #(mem/serialize %1 %2 scope) args arg-types)) + (apply downcall (map #(mem/serialize %1 %2 session) args arg-types)) ret-type))) (fn native-fn [& args] - (with-open [scope (mem/stack-scope)] + (with-open [session (mem/stack-session)] (mem/deserialize-from - (apply downcall (mem/scope-allocator scope) - (map #(mem/serialize %1 %2 scope) args arg-types)) + (apply downcall (mem/session-allocator session) + (map #(mem/serialize %1 %2 session) args arg-types)) ret-type))))) (defn make-serde-varargs-wrapper @@ -536,30 +536,30 @@ (defn- upcall-serde-wrapper "Creates a function that wraps `f` which deserializes the arguments and - serializes the return type in the [[global-scope]]." + serializes the return type in the [[global-session]]." [f arg-types ret-type] (fn [& args] (mem/serialize (apply f (map mem/deserialize args arg-types)) ret-type - (mem/global-scope)))) + (mem/global-session)))) (defmethod mem/serialize* ::fn - [f [_fn arg-types ret-type & {:keys [raw-fn?]}] scope] + [f [_fn arg-types ret-type & {:keys [raw-fn?]}] session] (.upcallStub (Linker/nativeLinker) (cond-> f (not raw-fn?) (upcall-serde-wrapper arg-types ret-type) :always (upcall-handle arg-types ret-type)) (function-descriptor arg-types ret-type) - scope)) + session)) (defmethod mem/deserialize* ::fn [addr [_fn arg-types ret-type & {:keys [raw-fn?]}]] (when-not (mem/null? addr) (vary-meta (-> addr - (MemorySegment/ofAddress mem/pointer-size (mem/connected-scope)) + (MemorySegment/ofAddress mem/pointer-size (mem/connected-session)) (downcall-handle (function-descriptor arg-types ret-type)) (downcall-fn arg-types ret-type) (cond-> (not raw-fn?) (make-serde-wrapper arg-types ret-type))) @@ -614,7 +614,7 @@ (mem/serialize-into newval (.-type static-var) (.-seg static-var) - (mem/global-scope)) + (mem/global-session)) newval) (defn fswap! @@ -642,7 +642,7 @@ [symbol-or-addr type] (StaticVariable. (mem/as-segment (.address (ensure-symbol symbol-or-addr)) (mem/size-of type) - (mem/global-scope)) + (mem/global-session)) type (atom nil))) (defmacro defvar diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 10d0bd8..615b3bc 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -1,5 +1,5 @@ (ns coffi.mem - "Functions for managing native allocations, resource scopes, and (de)serialization. + "Functions for managing native allocations, memory sessions, and (de)serialization. For any new type to be implemented, three multimethods must be overriden, but which three depends on the native representation of the type. @@ -16,7 +16,7 @@ segments. When writing code that manipulates a segment, it's best practice to - use [[with-acquired]] on the [[segment-scope]] in order to ensure it won't be + use [[with-acquired]] on the [[segment-session]] in order to ensure it won't be released during its manipulation." (:require [clojure.set :as set] @@ -42,23 +42,50 @@ (set! *warn-on-reflection* true) -(defn stack-scope +(defn stack-session + "Constructs a new session for use only in this thread. + + The memory allocated within this session is cheap to allocate, like a native + stack." + ^MemorySession [] + (MemorySession/openConfined)) + +(defn ^:deprecated stack-scope "Constructs a new scope for use only in this thread. The memory allocated within this scope is cheap to allocate, like a native stack." ^MemorySession [] - (MemorySession/openConfined)) + (stack-session)) -(defn shared-scope +(defn shared-session + "Constructs a new shared memory session. + + This session can be shared across threads and memory allocated in it will only + be cleaned up once every thread accessing the session closes it." + ^MemorySession [] + (MemorySession/openShared)) + +(defn ^:deprecated shared-scope "Constructs a new shared scope. This scope can be shared across threads and memory allocated in it will only be cleaned up once every thread accessing the scope closes it." ^MemorySession [] - (MemorySession/openShared)) + (shared-session)) -(defn connected-scope +(defn connected-session + "Constructs a new memory session to reclaim all connected resources at once. + + The session may be shared across threads, and all resources created with it + will be cleaned up at the same time, when all references have been collected. + + This type of session cannot be closed, and therefore should not be created in + a [[with-open]] clause." + ^MemorySession [] + (MemorySession/openImplicit)) + +(defn ^:deprecated connected-scope "Constructs a new scope to reclaim all connected resources at once. The scope may be shared across threads, and all resources created with it will @@ -67,9 +94,19 @@ This type of scope cannot be closed, and therefore should not be created in a [[with-open]] clause." ^MemorySession [] - (MemorySession/openImplicit)) + (connected-session)) -(defn global-scope +(defn global-session + "Constructs the global session, which will never reclaim its resources. + + This session may be shared across threads, but is intended mainly in cases + where memory is allocated with [[alloc]] but is either never freed or whose + management is relinquished to a native library, such as when returned from a + callback." + ^MemorySession [] + (MemorySession/global)) + +(defn ^:deprecated global-scope "Constructs the global scope, which will never reclaim its resources. This scope may be shared across threads, but is intended mainly in cases where @@ -77,29 +114,43 @@ management is relinquished to a native library, such as when returned from a callback." ^MemorySession [] - (MemorySession/global)) + (global-session)) -(defn scope-allocator +(defn session-allocator + "Constructs a segment allocator from the given `session`. + + This is primarily used when working with unwrapped downcall functions. When a + downcall function returns a non-primitive type, it must be provided with an + allocator." + ^SegmentAllocator [^MemorySession session] + (SegmentAllocator/newNativeArena session)) + +(defn ^:deprecated scope-allocator "Constructs a segment allocator from the given `scope`. This is primarily used when working with unwrapped downcall functions. When a downcall function returns a non-primitive type, it must be provided with an allocator." ^SegmentAllocator [^MemorySession scope] - (SegmentAllocator/newNativeArena scope)) + (session-allocator scope)) -(defn segment-scope - "Gets the scope used to construct the `segment`." +(defn segment-session + "Gets the memory session used to construct the `segment`." ^MemorySession [segment] (.session ^MemorySegment segment)) +(defn ^:deprecated segment-scope + "Gets the scope used to construct the `segment`." + ^MemorySession [segment] + (segment-session segment)) + (defn alloc "Allocates `size` bytes. - If a `scope` is provided, the allocation will be reclaimed when it is closed." - (^MemorySegment [size] (alloc size (connected-scope))) - (^MemorySegment [size scope] (MemorySegment/allocateNative (long size) ^MemorySession scope)) - (^MemorySegment [size alignment scope] (MemorySegment/allocateNative (long size) (long alignment) ^MemorySession scope))) + If a `session` is provided, the allocation will be reclaimed when it is closed." + (^MemorySegment [size] (alloc size (connected-session))) + (^MemorySegment [size session] (MemorySegment/allocateNative (long size) ^MemorySession session)) + (^MemorySegment [size alignment session] (MemorySegment/allocateNative (long size) (long alignment) ^MemorySession session))) (defn alloc-with "Allocates `size` bytes using the `allocator`." @@ -109,23 +160,23 @@ (.allocate ^SegmentAllocator allocator (long size) (long alignment)))) (defmacro with-acquired - "Acquires one or more `scopes` until the `body` completes. + "Acquires one or more `sessions` until the `body` completes. - This is only necessary to do on shared scopes, however if you are operating on - an arbitrary passed scope, it is best practice to wrap code that interacts - with it wrapped in this." + This is only necessary to do on shared sessions, however if you are operating + on an arbitrary passed session, it is best practice to wrap code that + interacts with it wrapped in this." {:style/indent 1} - [scopes & body] - (if (seq scopes) - `(let [scope# ~(first scopes)] + [sessions & body] + (if (seq sessions) + `(let [session# ~(first sessions)] (.whileAlive - ^MemorySession scope# + ^MemorySession session# (^:once fn* [] - (with-acquired [~@(rest scopes)] + (with-acquired [~@(rest sessions)] ~@body)))) `(do ~@body))) (s/fdef with-acquired - :args (s/cat :scopes any? + :args (s/cat :sessions any? :body (s/* any?))) (defn address-of @@ -167,33 +218,33 @@ (.addOffset ^MemoryAddress address (long offset))) (defn add-close-action! - "Adds a 0-arity function to be run when the `scope` closes." - [^MemorySession scope ^Runnable action] - (.addCloseAction scope action) + "Adds a 0-arity function to be run when the `session` closes." + [^MemorySession session ^Runnable action] + (.addCloseAction session action) nil) (defn as-segment - "Dereferences an `address` into a memory segment associated with the `scope`." + "Dereferences an `address` into a memory segment associated with the `session`." (^MemorySegment [^MemoryAddress address size] - (MemorySegment/ofAddress address (long size) (connected-scope))) - (^MemorySegment [^MemoryAddress address size scope] - (MemorySegment/ofAddress address (long size) scope))) + (MemorySegment/ofAddress address (long size) (connected-session))) + (^MemorySegment [^MemoryAddress address size session] + (MemorySegment/ofAddress address (long size) session))) (defn copy-segment "Copies the content to `dest` from `src`. Returns `dest`." ^MemorySegment [^MemorySegment dest ^MemorySegment src] - (with-acquired (map segment-scope [src dest]) + (with-acquired (map segment-session [src dest]) (.copyFrom dest src) dest)) (defn clone-segment "Clones the content of `segment` into a new segment of the same size." - (^MemorySegment [segment] (clone-segment segment (connected-scope))) - (^MemorySegment [^MemorySegment segment scope] - (with-acquired [(segment-scope segment) scope] - (copy-segment ^MemorySegment (alloc (.byteSize segment) scope) segment)))) + (^MemorySegment [segment] (clone-segment segment (connected-session))) + (^MemorySegment [^MemorySegment segment session] + (with-acquired [(segment-session segment) session] + (copy-segment ^MemorySegment (alloc (.byteSize segment) session) segment)))) (defn slice-segments "Constructs a lazy seq of `size`-length memory segments, sliced from `segment`." @@ -835,8 +886,8 @@ (defn alloc-instance "Allocates a memory segment for the given `type`." - (^MemorySegment [type] (alloc-instance type (connected-scope))) - (^MemorySegment [type scope] (MemorySegment/allocateNative ^long (size-of type) ^MemorySession scope))) + (^MemorySegment [type] (alloc-instance type (connected-session))) + (^MemorySegment [type session] (MemorySegment/allocateNative ^long (size-of type) ^MemorySession session))) (declare serialize serialize-into) @@ -844,68 +895,68 @@ "Constructs a serialized version of the `obj` and returns it. Any new allocations made during the serialization should be tied to the given - `scope`, except in extenuating circumstances. + `session`, except in extenuating circumstances. This method should only be implemented for types that serialize to primitives." (fn #_{:clj-kondo/ignore [:unused-binding]} - [obj type scope] + [obj type session] (type-dispatch type))) (defmethod serialize* :default - [obj type _scope] + [obj type _session] (throw (ex-info "Attempted to serialize a non-primitive type with primitive methods" {:type type :object obj}))) (defmethod serialize* ::byte - [obj _type _scope] + [obj _type _session] (byte obj)) (defmethod serialize* ::short - [obj _type _scope] + [obj _type _session] (short obj)) (defmethod serialize* ::int - [obj _type _scope] + [obj _type _session] (int obj)) (defmethod serialize* ::long - [obj _type _scope] + [obj _type _session] (long obj)) (defmethod serialize* ::char - [obj _type _scope] + [obj _type _session] (char obj)) (defmethod serialize* ::float - [obj _type _scope] + [obj _type _session] (float obj)) (defmethod serialize* ::double - [obj _type _scope] + [obj _type _session] (double obj)) (defmethod serialize* ::pointer - [obj type scope] + [obj type session] (if-not (null? obj) (if (sequential? type) - (with-acquired [scope] - (let [segment (alloc-instance (second type) scope)] - (serialize-into obj (second type) segment scope) + (with-acquired [session] + (let [segment (alloc-instance (second type) session)] + (serialize-into obj (second type) segment session) (address-of segment))) obj) (MemoryAddress/NULL))) (defmethod serialize* ::void - [_obj _type _scope] + [_obj _type _session] nil) (defmulti serialize-into "Writes a serialized version of the `obj` to the given `segment`. Any new allocations made during the serialization should be tied to the given - `scope`, except in extenuating circumstances. + `session`, except in extenuating circumstances. This method should be implemented for any type which does not override [[c-layout]]. @@ -914,66 +965,66 @@ the result value into the `segment`. Implementations of this should be inside a [[with-acquired]] block for the - `scope` if they perform multiple memory operations." + `session` if they perform multiple memory operations." (fn #_{:clj-kondo/ignore [:unused-binding]} - [obj type segment scope] + [obj type segment session] (type-dispatch type))) (defmethod serialize-into :default - [obj type segment scope] + [obj type segment session] (if-some [prim-layout (primitive-type type)] - (with-acquired [(segment-scope segment) scope] - (serialize-into (serialize* obj type scope) prim-layout segment scope)) + (with-acquired [(segment-session segment) session] + (serialize-into (serialize* obj type session) prim-layout segment session)) (throw (ex-info "Attempted to serialize an object to a type that has not been overriden" {:type type :object obj})))) (defmethod serialize-into ::byte - [obj _type segment _scope] + [obj _type segment _session] (write-byte segment (byte obj))) (defmethod serialize-into ::short - [obj type segment _scope] + [obj type segment _session] (if (sequential? type) (write-short segment 0 (second type) (short obj)) (write-short segment (short obj)))) (defmethod serialize-into ::int - [obj type segment _scope] + [obj type segment _session] (if (sequential? type) (write-int segment 0 (second type) (int obj)) (write-int segment (int obj)))) (defmethod serialize-into ::long - [obj type segment _scope] + [obj type segment _session] (if (sequential? type) (write-long segment 0 (second type) (long obj)) (write-long segment (long obj)))) (defmethod serialize-into ::char - [obj _type segment _scope] + [obj _type segment _session] (write-char segment (char obj))) (defmethod serialize-into ::float - [obj type segment _scope] + [obj type segment _session] (if (sequential? type) (write-float segment 0 (second type) (float obj)) (write-float segment (float obj)))) (defmethod serialize-into ::double - [obj type segment _scope] + [obj type segment _session] (if (sequential? type) (write-double segment 0 (second type) (double obj)) (write-double segment (double obj)))) (defmethod serialize-into ::pointer - [obj type segment scope] - (with-acquired [(segment-scope segment) scope] + [obj type segment session] + (with-acquired [(segment-session segment) session] (write-address segment (cond-> obj - (sequential? type) (serialize* type scope))))) + (sequential? type) (serialize* type session))))) (defn serialize "Serializes an arbitrary type. @@ -981,12 +1032,12 @@ For types which have a primitive representation, this serializes into that representation. For types which do not, it allocates a new segment and serializes into that." - ([obj type] (serialize obj type (connected-scope))) - ([obj type scope] + ([obj type] (serialize obj type (connected-session))) + ([obj type session] (if (primitive-type type) - (serialize* obj type scope) - (let [segment (alloc-instance type scope)] - (serialize-into obj type segment scope) + (serialize* obj type session) + (let [segment (alloc-instance type session)] + (serialize-into obj type segment session) segment)))) (declare deserialize deserialize*) @@ -998,7 +1049,7 @@ deserialize the primitive before calling [[deserialize*]]. Implementations of this should be inside a [[with-acquired]] block for the the - `segment`'s scope if they perform multiple memory operations." + `segment`'s session if they perform multiple memory operations." (fn #_{:clj-kondo/ignore [:unused-binding]} [segment type] @@ -1054,7 +1105,7 @@ (defmethod deserialize-from ::pointer [segment type] - (with-acquired [(segment-scope segment)] + (with-acquired [(segment-session segment)] (cond-> (read-address segment) (sequential? type) (deserialize* type)))) @@ -1129,7 +1180,7 @@ (defn seq-of "Constructs a lazy sequence of `type` elements deserialized from `segment`." [type segment] - (with-acquired [(segment-scope segment)] + (with-acquired [(segment-session segment)] (map #(deserialize % type) (slice-segments segment (size-of type))))) ;;; Raw composite types @@ -1141,7 +1192,7 @@ (c-layout type)) (defmethod serialize-into ::raw - [obj _type segment _scope] + [obj _type segment _session] (copy-segment segment obj)) (defmethod deserialize-from ::raw @@ -1155,9 +1206,9 @@ ::pointer) (defmethod serialize* ::c-string - [obj _type scope] + [obj _type session] (if obj - (address-of (.allocateUtf8String (scope-allocator scope) ^String obj)) + (address-of (.allocateUtf8String (session-allocator session) ^String obj)) (MemoryAddress/NULL))) (defmethod deserialize* ::c-string @@ -1174,7 +1225,7 @@ (into-array MemoryLayout items)))) (defmethod serialize-into ::union - [obj [_union _types & {:keys [dispatch extract]} :as type] segment scope] + [obj [_union _types & {:keys [dispatch extract]} :as type] segment session] (when-not dispatch (throw (ex-info "Attempted to serialize a union with no dispatch function" {:type type @@ -1186,7 +1237,7 @@ obj) type segment - scope))) + session))) (defmethod deserialize-from ::union [segment type] @@ -1203,7 +1254,7 @@ (into-array MemoryLayout fields)))) (defmethod serialize-into ::struct - [obj [_struct fields] segment scope] + [obj [_struct fields] segment session] (loop [offset 0 fields fields] (when (seq fields) @@ -1211,7 +1262,7 @@ size (size-of type)] (serialize-into (get obj field) type - (slice segment offset size) scope) + (slice segment offset size) session) (recur (long (+ offset size)) (rest fields)))))) (defmethod deserialize-from ::struct @@ -1237,7 +1288,7 @@ (MemoryLayout/paddingLayout (* 8 size))) (defmethod serialize-into ::padding - [_obj [_padding _size] _segment _scope] + [_obj [_padding _size] _segment _session] nil) (defmethod deserialize-from ::padding @@ -1253,9 +1304,9 @@ (c-layout type))) (defmethod serialize-into ::array - [obj [_array type count] segment scope] + [obj [_array type count] segment session] (dorun - (map #(serialize-into %1 type %2 scope) + (map #(serialize-into %1 type %2 session) obj (slice-segments (slice segment 0 (* count (size-of type))) (size-of type))))) @@ -1300,10 +1351,10 @@ variants)))) (defmethod serialize* ::enum - [obj [_enum variants & {:keys [repr]}] scope] + [obj [_enum variants & {:keys [repr]}] session] (serialize* ((enum-variants-map variants) obj) (or repr ::int) - scope)) + session)) (defmethod deserialize* ::enum [obj [_enum variants & {:keys [_repr]}]] @@ -1318,9 +1369,9 @@ ::int)) (defmethod serialize* ::flagset - [obj [_flagset bits & {:keys [repr]}] scope] + [obj [_flagset bits & {:keys [repr]}] session] (let [bits-map (enum-variants-map bits)] - (reduce #(bit-set %1 (get bits-map %2)) (serialize* 0 (or repr ::int) scope) obj))) + (reduce #(bit-set %1 (get bits-map %2)) (serialize* 0 (or repr ::int) session) obj))) (defmethod deserialize* ::flagset [obj [_flagset bits & {:keys [repr]}]] @@ -1352,8 +1403,8 @@ [_type#] (primitive-type aliased#)) (defmethod serialize* ~new-type - [obj# _type# scope#] - (serialize* obj# aliased# scope#)) + [obj# _type# session#] + (serialize* obj# aliased# session#)) (defmethod deserialize* ~new-type [obj# _type#] (deserialize* obj# aliased#))) @@ -1362,8 +1413,8 @@ [_type#] (c-layout aliased#)) (defmethod serialize-into ~new-type - [obj# _type# segment# scope#] - (serialize-into obj# aliased# segment# scope#)) + [obj# _type# segment# session#] + (serialize-into obj# aliased# segment# session#)) (defmethod deserialize-from ~new-type [segment# _type#] (deserialize-from segment# aliased#))))) From 177078e83e7a3216a29d1155b903ef85c08a75a7 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 11 Nov 2022 14:09:20 -0600 Subject: [PATCH 26/50] Update changelog for JDK 19 support --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index d91551b..737fefa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,7 @@ All notable changes to this project will be documented in this file. This change ## [Unreleased] ### Added +- Support for JDK 19 - New macros for defining vars with values from native code - New function to allow getting the backing memory segment of a `coffi.ffi.StaticVariable`, to replace the `Addressable` implementation lost in the migration to JDK 18 @@ -12,6 +13,9 @@ All notable changes to this project will be documented in this file. This change - Bug where padding in structs may be increased when fields have alignments less than their size - Bug where pointer alignment was incorrectly defined +### Changed +- References to `scope` as a term have been changed to `session` to match Panama messaging. Where this conflicts with function names, old versions have been deprecated and new names have been introduced. + ## [0.5.357] - 2022-07-07 ### Removed - `:coffi.mem/long-long` primitive type From 8a7b9cea2ccbe4c0ed84fd7edd5d8995a309372b Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 11 Nov 2022 14:40:52 -0600 Subject: [PATCH 27/50] Update readme for JDK 19 support --- README.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index d574274..f15cfc7 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ Coffi is a foreign function interface library for Clojure, using the new [Project Panama](https://openjdk.java.net/projects/panama/) that's a part of the -incubator in Java 18. This allows calling native code directly from Clojure +incubator in Java 19. This allows calling native code directly from Clojure without the need for either Java or native code specific to the library, as e.g. the JNI does. Coffi focuses on ease of use, including functions and macros for creating wrappers to allow the resulting native functions to act just like @@ -1129,11 +1129,9 @@ the latest JDK. If a particular feature is not specific to the newer JDK, PRs backporting it to versions of coffi supporting Java 17 will likely be accepted. ### 1.0 Release -Because the feature that coffi wraps in the JDK is an incubator feature (and -likely in JDK 19 a [preview -feature](https://mail.openjdk.java.net/pipermail/panama-dev/2021-September/014946.html)) -coffi itself will not be released in a 1.0.x version until the feature becomes a -core part of the JDK, likely before or during the next LTS release, Java 21, in +Because the feature that coffi wraps in the JDK is in preview as of JDK 19, coffi +itself will not be released in a 1.0.x version until the feature becomes a core +part of the JDK, likely before or during the next LTS release, Java 21, in September 2023. ## License From 4c65b726e92f16a9d09021b118da37140a6f583e Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 11 Nov 2022 15:50:29 -0600 Subject: [PATCH 28/50] Add note about M1 bug --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f15cfc7..337c921 100644 --- a/README.md +++ b/README.md @@ -1103,7 +1103,10 @@ stands, coffi is the fastest FFI available to Clojure developers. The project author is aware of these issues and plans to fix them in a future release: -There are currently no known issues! Hooray! +- On M1 Macs occasional crashes are present when returning structs by value from + native code. At the moment this appears to be an upstream issue with Panama, + and will be reported once a minimal reproduction case with only Panama is + produced. ## Future Plans These features are planned for future releases. From 85a82f3d949aad3d0b74987782f913c964feb534 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Mon, 14 Nov 2022 10:55:15 -0600 Subject: [PATCH 29/50] Update readme to indicate the current versions for each JDK --- README.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 337c921..c22cfa4 100644 --- a/README.md +++ b/README.md @@ -1125,16 +1125,17 @@ These features are planned for future releases. The purpose of coffi is to provide a wrapper for published versions of Project Panama, starting with JDK 17. As new JDKs are released, coffi will be ported to the newer versions of Panama. Version `0.4.341` is the last version compatible -with JDK 17. Bugfixes, and potential backports of newer coffi features may be -found on the `jdk17-lts` branch. Development of new features and fixes as well -as support for new Panama idioms and features will continue with focus only on -the latest JDK. If a particular feature is not specific to the newer JDK, PRs -backporting it to versions of coffi supporting Java 17 will likely be accepted. +with JDK 17. Version `0.5.357` is the last version compatible with JDK 18. +Bugfixes, and potential backports of newer coffi features may be found on the +`jdk17-lts` branch. Development of new features and fixes as well as support for +new Panama idioms and features will continue with focus only on the latest JDK. +If a particular feature is not specific to the newer JDK, PRs backporting it to +versions of coffi supporting Java 17 will likely be accepted. ### 1.0 Release -Because the feature that coffi wraps in the JDK is in preview as of JDK 19, coffi -itself will not be released in a 1.0.x version until the feature becomes a core -part of the JDK, likely before or during the next LTS release, Java 21, in +Because the feature that coffi wraps in the JDK is in preview as of JDK 19, +coffi itself will not be released in a 1.0.x version until the feature becomes a +core part of the JDK, likely before or during the next LTS release, Java 21, in September 2023. ## License From 0e23a514f096d8d6fe840a62b78195593aa85ef8 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Wed, 16 Nov 2022 09:27:01 -0600 Subject: [PATCH 30/50] Fix misspelling in ex-info message --- src/clj/coffi/mem.clj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 615b3bc..b9880fe 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -976,7 +976,7 @@ (if-some [prim-layout (primitive-type type)] (with-acquired [(segment-session segment) session] (serialize-into (serialize* obj type session) prim-layout segment session)) - (throw (ex-info "Attempted to serialize an object to a type that has not been overriden" + (throw (ex-info "Attempted to serialize an object to a type that has not been overridden" {:type type :object obj})))) From 144889bc953f020d758f1387a7dcf0dc4bd002fb Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Wed, 16 Nov 2022 09:36:54 -0600 Subject: [PATCH 31/50] Update readme for new version of coffi --- README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index c22cfa4..1190401 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ Coffi is a foreign function interface library for Clojure, using the new [Project Panama](https://openjdk.java.net/projects/panama/) that's a part of the -incubator in Java 19. This allows calling native code directly from Clojure +preview in Java 19. This allows calling native code directly from Clojure without the need for either Java or native code specific to the library, as e.g. the JNI does. Coffi focuses on ease of use, including functions and macros for creating wrappers to allow the resulting native functions to act just like @@ -27,19 +27,19 @@ library. $ clj -X:deps prep ``` -Coffi requires usage of the module `jdk.incubator.foreign`, which means that the -JVM must enable the usage of this module. In order to use coffi, add the -following JVM arguments to your application. +Coffi requires usage of the package `java.lang.foreign`, and everything in this +package is considered to be a preview release, which are disabled by default. In +order to use coffi, add the following JVM arguments to your application. ```sh ---add-modules=jdk.incubator.foreign --enable-native-access=ALL-UNNAMED +--enable-preview --enable-native-access=ALL-UNNAMED ``` You can specify JVM arguments in a particular invocation of the Clojure CLI with the -J flag like so: ``` sh -clj -J--add-modules=jdk.incubator.foreign -J--enable-native-access=ALL-UNNAMED +clj -J--enable-preview -J--enable-native-access=ALL-UNNAMED ``` You can also specify them in an alias in your `deps.edn` file under the @@ -47,7 +47,7 @@ You can also specify them in an alias in your `deps.edn` file under the using `-M`, `-A`, or `-X`. ``` clojure -{:aliases {:dev {:jvm-opts ["--add-modules=jdk.incubator.foreign" "--enable-native-access=ALL-UNNAMED"]}}} +{:aliases {:dev {:jvm-opts ["--enable-preview" "--enable-native-access=ALL-UNNAMED"]}}} ``` Other build tools should provide similar functionality if you check their @@ -581,7 +581,7 @@ With raw handles, the argument types are expected to exactly match the types expected by the native function. For primitive types, those are primitives. For addresses, that is `MemoryAddress`, and for composite types like structs and unions, that is `MemorySegment`. Both `MemoryAddress` and `MemorySegment` come -from the `jdk.incubator.foreign` package. +from the `java.lang.foreign` package. In addition, when a raw handle returns a composite type represented with a `MemorySegment`, it requires an additional first argument, a `SegmentAllocator`, @@ -726,6 +726,8 @@ appealing, as they have a smaller API surface area and it's easier to wrap functions. ### Benchmarks +**BENCHMARKS FOR COFFI AND DTYPE-NEXT ARE BASED ON AN OLD VERSION. NEW BENCHMARKS WILL BE CREATED WHEN PANAMA COMES OUT OF PREVIEW** + An additional consideration when thinking about alternatives is the performance of each available option. It's an established fact that JNA (used by all three alternative libraries on JDK <16) introduces more overhead when calling native @@ -968,8 +970,6 @@ coming from, but I'll admit that I haven't looked at their implementations very closely. #### dtype-next -**BENCHMARKS FOR DTYPE-NEXT ARE BASED ON AN OLD VERSION. NEW BENCHMARKS WILL BE COMING SHORTLY** - The library dtype-next replaced tech.jna in the toolkit of the group working on machine learning and array-based programming, and it includes support for composite data types including structs, as well as primitive functions and From 857da5949aef12b6903d08f847063125306c91db Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Wed, 16 Nov 2022 16:57:30 -0600 Subject: [PATCH 32/50] Rename scope to session in readme --- README.md | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index 1190401..53b4fc4 100644 --- a/README.md +++ b/README.md @@ -345,29 +345,33 @@ This can be used to implement out variables often seen in native code. (deserialize int-ptr [::mem/pointer ::mem/int]))) ``` -### Scopes +### Sessions +**Before JDK 19 Sessions were called Scopes. Coffi retains functions that are +named for creating scopes for backwards compatibility, but they will be removed +in version 1.0.** + In order to serialize any non-primitive type (such as the previous `[::mem/pointer ::mem/int]`), off-heap memory needs to be allocated. When memory -is allocated inside the JVM, the memory is associated with a scope. Because none -was provided here, the scope is an implicit scope, and the memory will be freed -when the serialized object is garbage collected. +is allocated inside the JVM, the memory is associated with a session. Because +none was provided here, the session is an implicit session, and the memory will +be freed when the serialized object is garbage collected. In many cases this is not desirable, because the memory is not freed in a deterministic manner, causing garbage collection pauses to become longer, as -well as changing allocation performance. Instead of an implicit scope, there are -other kinds of scopes as well. A `stack-scope` is a thread-local scope. Stack -scopes are `Closeable`, which means they should usually be used in a `with-open` -form. When a `stack-scope` is closed, it immediately frees all the memory -associated with it. The previous example, `out-int`, can be implemented with a -stack scope. +well as changing allocation performance. Instead of an implicit session, there +are other kinds of sessions as well. A `stack-session` is a thread-local +session. Stack sessions are `Closeable`, which means they should usually be used +in a `with-open` form. When a `stack-session` is closed, it immediately frees +all the memory associated with it. The previous example, `out-int`, can be +implemented with a stack session. ```clojure (defcfn out-int "out_int" [::mem/pointer] ::mem/void native-fn [i] - (with-open [scope (mem/stack-scope)] - (let [int-ptr (mem/serialize i [::mem/pointer ::mem/int] scope)] + (with-open [session (mem/stack-session)] + (let [int-ptr (mem/serialize i [::mem/pointer ::mem/int] session)] (native-fn int-ptr) (mem/deserialize int-ptr [::mem/pointer ::mem/int])))) ``` @@ -375,14 +379,15 @@ stack scope. This will free the pointer immediately upon leaving the function. When memory needs to be accessible from multiple threads, there's -`shared-scope`. When using a `shared-scope`, it should be accessed inside a -`with-acquired` block. When a `shared-scope` is `.close`d, it will release all +`shared-session`. When using a `shared-session`, it should be accessed inside a +`with-acquired` block. When a `shared-session` is `.close`d, it will release all its associated memory when every `with-acquired` block associated with it is exited. -In addition, two non-`Closeable` scopes are `global-scope`, which never frees -the resources associated with it, and `connected-scope`, which is a scope that -frees its resources on garbage collection, like an implicit scope. +In addition, two non-`Closeable` sessions are `global-session`, which never +frees the resources associated with it, and `connected-session`, which is a +session that frees its resources on garbage collection, like an implicit +session. ### Serialization and Deserialization Custom serializers and deserializers may also be created. This is done using two From 1ab231dee59108a0a5c2c533a208a8c1f396432c Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Wed, 16 Nov 2022 16:57:52 -0600 Subject: [PATCH 33/50] Remove a reference to long-long --- README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 53b4fc4..1f65f22 100644 --- a/README.md +++ b/README.md @@ -132,14 +132,13 @@ Coffi defines a basic set of primitive types: Each of these types maps to their C counterpart. Values of any of these primitive types except for `pointer` will be cast with their corresponding -Clojure function (with `long-long` mapping to the `long` function) when they are -passed as arguments to native functions. Additionally, the `c-string` type is -defined, although it is not primitive. +Clojure function when they are passed as arguments to native functions. +Additionally, the `c-string` type is defined, although it is not primitive. ### Composite Types In addition, some composite types are also defined in coffi, including struct and union types (unions will be discussed with serialization and -deserialization). For an example c struct and function: +deserialization). For an example C struct and function: ```c typedef struct point { From 139341af9948efbc1dde0cfd01a050b0f70fe374 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Wed, 16 Nov 2022 16:57:58 -0600 Subject: [PATCH 34/50] Add a paragraph about catching exceptions in upcalls --- README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/README.md b/README.md index 1f65f22..d793898 100644 --- a/README.md +++ b/README.md @@ -238,6 +238,16 @@ type and message in the registers section, but it's important to be aware of all the same. Ideally you should test your callbacks before actually passing them to native code. +When writing a wrapper library for a C library, it may be a good choice to wrap +all passed Clojure functions in an additional function which catches all +throwables, potentially notifies the user in some manner (e.g. logging), and +returns a default value. This is on the wrapper library's developer to decide +when and where this is appropriate, as in some cases no reasonable default +return value can be determined and it is most sensible to simply crash the JVM. +This is the reason that coffi defaults to this behavior, as in the author's +opinion it is better to fail hard and fast rather than to attempt to produce a +default and cause unexpected behavior later. + Another important thing to keep in mind is the expected lifetime of the function that you pass to native code. For example it is perfectly fine to pass an anonymous function to a native function if the callback will never be called From 65c7544cc7ed9d6c815da821a9a9899cf185b3e4 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Wed, 16 Nov 2022 16:58:20 -0600 Subject: [PATCH 35/50] Add extra constructors for sessions --- src/clj/coffi/mem.clj | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index b9880fe..dd3372a 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -38,6 +38,7 @@ ValueLayout$OfFloat ValueLayout$OfDouble ValueLayout$OfAddress) + (java.lang.ref Cleaner) (java.nio ByteOrder))) (set! *warn-on-reflection* true) @@ -47,8 +48,10 @@ The memory allocated within this session is cheap to allocate, like a native stack." - ^MemorySession [] - (MemorySession/openConfined)) + (^MemorySession [] + (MemorySession/openConfined)) + (^MemorySession [^Cleaner cleaner] + (MemorySession/openConfined cleaner))) (defn ^:deprecated stack-scope "Constructs a new scope for use only in this thread. @@ -63,8 +66,10 @@ This session can be shared across threads and memory allocated in it will only be cleaned up once every thread accessing the session closes it." - ^MemorySession [] - (MemorySession/openShared)) + (^MemorySession [] + (MemorySession/openShared)) + (^MemorySession [^Cleaner cleaner] + (MemorySession/openShared cleaner))) (defn ^:deprecated shared-scope "Constructs a new shared scope. From 12b766129562d8278175d34fc91bfa2de51dd3c6 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 29 Nov 2022 13:41:55 -0600 Subject: [PATCH 36/50] Update readme to use session rather than scope --- README.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index d793898..eb26610 100644 --- a/README.md +++ b/README.md @@ -426,8 +426,8 @@ serialize to primitives. ```clojure (defmethod mem/serialize* ::vector - [obj _type scope] - (mem/address-of (mem/serialize obj [::mem/array ::mem/float 3] scope))) + [obj _type session] + (mem/address-of (mem/serialize obj [::mem/array ::mem/float 3] session))) (defmethod mem/deserialize* ::vector [addr _type] @@ -436,9 +436,9 @@ serialize to primitives. ``` The `slice-global` function allows you to take an address without an associated -scope and get a memory segment which can be deserialized. +session and get a memory segment which can be deserialized. -In cases like this where we don't know the scope of the pointer, we could use +In cases like this where we don't know the session of the pointer, we could use `add-close-action!` to ensure it's freed. For example if a `free-vector!` function that takes a pointer exists, we could use this: @@ -446,14 +446,14 @@ function that takes a pointer exists, we could use this: (defcfn returns-vector "returns_vector" [] ::mem/pointer native-fn - [scope] + [session] (let [ret-ptr (native-fn)] - (add-close-action! scope #(free-vector! ret-ptr)) + (add-close-action! session #(free-vector! ret-ptr)) (deserialize ret-ptr ::vector))) ``` -This function takes a scope and returns the deserialized vector, and it will -free the pointer when the scope closes. +This function takes a session and returns the deserialized vector, and it will +free the pointer when the session closes. #### Tagged Union For the tagged union type, we will represent the value as a vector of a keyword @@ -505,7 +505,7 @@ deserialize the value into and out of memory segments. This is accomplished with (map first)))) (defmethod mem/serialize-into ::tagged-union - [obj [_tagged-union tags type-map] segment scope] + [obj [_tagged-union tags type-map] segment session] (mem/serialize-into {:tag (item-index tags (first obj)) :value (second obj)} @@ -513,7 +513,7 @@ deserialize the value into and out of memory segments. This is accomplished with [[:tag ::mem/long] [:value (get type-map (first obj))]]] segment - scope)) + session)) ``` This serialization method is rather simple, it just turns the vector value into @@ -570,7 +570,7 @@ it could be represented for serialization purposes like so: This union however would not include the tag when serialized. If a union is deserialized, then all that coffi does is to allocate a new -segment of the appropriate size with an implicit scope so that it may later be +segment of the appropriate size with an implicit session so that it may later be garbage collected, and copies the data from the source segment into it. It's up to the user to call `deserialize-from` on that segment with the appropriate type. @@ -599,8 +599,8 @@ from the `java.lang.foreign` package. In addition, when a raw handle returns a composite type represented with a `MemorySegment`, it requires an additional first argument, a `SegmentAllocator`, -which can be acquired with `scope-allocator` to get one associated with a -specific scope. The returned value will live until that scope is released. +which can be acquired with `session-allocator` to get one associated with a +specific session. The returned value will live until that session is released. In addition, function types can be specified as being raw, in the following manner: @@ -640,8 +640,8 @@ floats, the following code might be used. (defn returns-float-array [] - (with-open [scope (mem/stack-scope)] - (let [out-floats (mem/alloc mem/pointer-size scope) + (with-open [session (mem/stack-session)] + (let [out-floats (mem/alloc mem/pointer-size session) num-floats (function-handle (mem/address-of out-floats)) floats-addr (mem/read-address out-floats) floats-slice (mem/slice-global floats-addr (unchecked-multiply-int mem/float-size num-floats))] From 4c60ff5085f2572271c7626e62869b3f906a350a Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 29 Nov 2022 13:43:06 -0600 Subject: [PATCH 37/50] Add test for global variables --- test/c/ffi_test.c | 2 +- test/clj/coffi/ffi_test.clj | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/test/c/ffi_test.c b/test/c/ffi_test.c index 8235c1b..c689e87 100644 --- a/test/c/ffi_test.c +++ b/test/c/ffi_test.c @@ -26,7 +26,7 @@ CString upcall_test(StringFactory fun) { return fun(); } -static int counter = 0; +int counter = 0; static char* responses[] = { "Hello, world!", "Goodbye friend.", "co'oi prenu" }; diff --git a/test/clj/coffi/ffi_test.clj b/test/clj/coffi/ffi_test.clj index d7be985..15cc4b6 100644 --- a/test/clj/coffi/ffi_test.clj +++ b/test/clj/coffi/ffi_test.clj @@ -44,3 +44,8 @@ {:a \x :x 3.14 :y 42.0}))) + +(t/deftest static-variables-are-mutable + (ffi/freset! (ffi/static-variable "counter" ::mem/int) 1) + (t/is (= ((ffi/cfn "get_string1" [] ::mem/c-string)) + "Goodbye friend."))) From 379dc95c66e971ceff6f31eb69a9336b13e1f9ac Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 29 Nov 2022 18:52:11 -0600 Subject: [PATCH 38/50] Fix dumb implementation of copy-segment --- src/clj/coffi/mem.clj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index dd3372a..337bf36 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -240,7 +240,7 @@ Returns `dest`." ^MemorySegment [^MemorySegment dest ^MemorySegment src] - (with-acquired (map segment-session [src dest]) + (with-acquired [(segment-session src) (segment-session dest)] (.copyFrom dest src) dest)) From 8401cc3add7ac24d1c50e675a5ca1d1868845171 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 29 Nov 2022 20:06:20 -0600 Subject: [PATCH 39/50] Fix bug where `with-acquired` did not return a value --- src/clj/coffi/mem.clj | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 337bf36..1f171dc 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -173,12 +173,14 @@ {:style/indent 1} [sessions & body] (if (seq sessions) - `(let [session# ~(first sessions)] + `(let [session# ~(first sessions) + res# (volatile! ::invalid-value)] (.whileAlive ^MemorySession session# (^:once fn* [] (with-acquired [~@(rest sessions)] - ~@body)))) + (vreset! res# (do ~@body))))) + @res#) `(do ~@body))) (s/fdef with-acquired :args (s/cat :sessions any? From 955063b1ba224f394a9cdec974d1340cac670589 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 29 Nov 2022 20:06:44 -0600 Subject: [PATCH 40/50] Update the clojure dep --- deps.edn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deps.edn b/deps.edn index d410600..ffd8731 100644 --- a/deps.edn +++ b/deps.edn @@ -1,5 +1,5 @@ {:paths ["src/clj" "target/classes" "resources"] - :deps {org.clojure/clojure {:mvn/version "1.10.3"} + :deps {org.clojure/clojure {:mvn/version "1.11.1"} insn/insn {:mvn/version "0.2.1"}} :deps/prep-lib {:alias :build From 5ccf3694776b83c5ab2730d46b9380384bc97949 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 29 Nov 2022 20:21:34 -0600 Subject: [PATCH 41/50] Fix a dumb of my dumb --- src/clj/coffi/mem.clj | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 1f171dc..786c0aa 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -178,8 +178,9 @@ (.whileAlive ^MemorySession session# (^:once fn* [] - (with-acquired [~@(rest sessions)] - (vreset! res# (do ~@body))))) + (vreset! res# + (with-acquired [~@(rest sessions)] + ~@body)))) @res#) `(do ~@body))) (s/fdef with-acquired From 2c45d8e875f4426e1631f987fb4c63b0927604eb Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 29 Nov 2022 21:31:28 -0600 Subject: [PATCH 42/50] Add support for primitive types to raws --- src/clj/coffi/mem.clj | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/clj/coffi/mem.clj b/src/clj/coffi/mem.clj index 786c0aa..abafd54 100644 --- a/src/clj/coffi/mem.clj +++ b/src/clj/coffi/mem.clj @@ -1201,11 +1201,15 @@ (defmethod serialize-into ::raw [obj _type segment _session] - (copy-segment segment obj)) + (if (instance? MemorySegment obj) + (copy-segment segment obj) + obj)) (defmethod deserialize-from ::raw [segment _type] - (clone-segment segment)) + (if (instance? MemorySegment segment) + (clone-segment segment) + segment)) ;;; C String type From 218545feb74779cfe80786c90d486d3d5e68ed4b Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 28 Mar 2023 14:11:14 -0500 Subject: [PATCH 43/50] Fix bug where `defvar` and friends didn't work One function was private that didn't need to be since it was being used by the macros. This commit just makes it public. --- src/clj/coffi/ffi.clj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index a455fe9..7c300b3 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -186,7 +186,7 @@ [handle args ret] (insn/new-instance (downcall-class args ret) ^MethodHandle handle)) -(defn- ensure-symbol +(defn ensure-symbol "Returns the argument if it is a [[MemorySegment]], otherwise calls [[find-symbol]] on it." ^MemorySegment [symbol-or-addr] From f22cf8f81ddec70351d8659bccad46db2610b246 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 28 Mar 2023 14:24:39 -0500 Subject: [PATCH 44/50] Fix uses of `defvar` not compiling --- src/clj/coffi/ffi.clj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index 7c300b3..e720519 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -654,7 +654,7 @@ (def ~(:var-name args) ~@(when-let [doc (:docstring args)] (list doc)) - (static-variable symbol#))))) + (static-variable symbol# ~(:type args)))))) (s/fdef defvar :args ::defconst-args) From 5deac6493430c84321853f6449ef51b2d5fa4783 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 28 Mar 2023 14:25:01 -0500 Subject: [PATCH 45/50] Fix primitive static variables not deserializing --- src/clj/coffi/ffi.clj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/clj/coffi/ffi.clj b/src/clj/coffi/ffi.clj index e720519..371aeef 100644 --- a/src/clj/coffi/ffi.clj +++ b/src/clj/coffi/ffi.clj @@ -594,7 +594,7 @@ (deftype StaticVariable [seg type meta] IDeref (deref [_] - (mem/deserialize seg type)) + (mem/deserialize-from seg type)) IObj (withMeta [_ meta-map] From 0b323e0909d7a48e5d8cf86791c897a004d0a8c5 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Tue, 28 Mar 2023 14:26:10 -0500 Subject: [PATCH 46/50] Update changelog --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 737fefa..3df1f3b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ All notable changes to this project will be documented in this file. This change - New function to allow getting the backing memory segment of a `coffi.ffi.StaticVariable`, to replace the `Addressable` implementation lost in the migration to JDK 18 ### Fixed +- Bug where `static-variable`s with primitive types would not deserialize properly on `deref` +- Uses of `defvar` not compiling - Bug where nil values would not be correctly coerced to null pointers when passed to inlined functions - Bug where inline serde functions would fail on complex pointer types - Bug where padding in structs may be increased when fields have alignments less than their size From f97444639d02d6878eda0b538295197f99ae941f Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 31 Mar 2023 10:18:04 -0500 Subject: [PATCH 47/50] Version bump --- build.clj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.clj b/build.clj index 87d2cf0..bdbf2ba 100644 --- a/build.clj +++ b/build.clj @@ -17,7 +17,7 @@ [clojure.tools.build.api :as b])) (def lib-coord 'org.suskalo/coffi) -(def version (format "0.5.%s" (b/git-count-revs nil))) +(def version (format "0.6.%s" (b/git-count-revs nil))) (def resource-dirs ["resources/"]) From f19eabe81981ac68e4da9b9376f3403a92b32d4d Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 31 Mar 2023 10:22:05 -0500 Subject: [PATCH 48/50] Update changelog --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3df1f3b..6e1d77d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,7 +1,7 @@ # Change Log All notable changes to this project will be documented in this file. This change log follows the conventions of [keepachangelog.com](http://keepachangelog.com/). -## [Unreleased] +## [0.6.409] - 2023-03-31 ### Added - Support for JDK 19 - New macros for defining vars with values from native code From 9d6051236550967a00151244e9e0c5630e2944b1 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 31 Mar 2023 10:25:02 -0500 Subject: [PATCH 49/50] Update readme for new release tag --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index eb26610..908748e 100644 --- a/README.md +++ b/README.md @@ -16,8 +16,8 @@ This library is available on Clojars. Add one of the following entries to the `:deps` key of your `deps.edn`: ```clojure -org.suskalo/coffi {:mvn/version "0.5.357"} -io.github.IGJoshua/coffi {:git/tag "v0.5.357" :git/sha "a9e3ed0"} +org.suskalo/coffi {:mvn/version "0.6.409"} +io.github.IGJoshua/coffi {:git/tag "v0.6.409" :git/sha "f974446"} ``` If you use this library as a git dependency, you will need to prepare the From 58f1e69b00f2fa4c3a934c581b8666a18a22e9e1 Mon Sep 17 00:00:00 2001 From: Joshua Suskalo Date: Fri, 31 Mar 2023 10:26:19 -0500 Subject: [PATCH 50/50] Update codox documentation --- deps.edn | 4 +-- docs/coffi.ffi.html | 21 +++++------ docs/coffi.layout.html | 4 +-- docs/coffi.mem.html | 80 +++++++++++++++++++++++------------------- docs/index.html | 2 +- 5 files changed, 59 insertions(+), 52 deletions(-) diff --git a/deps.edn b/deps.edn index ffd8731..d0e1469 100644 --- a/deps.edn +++ b/deps.edn @@ -24,8 +24,8 @@ :codox {:extra-deps {codox/codox {:mvn/version "0.10.7"}} :exec-fn codox.main/generate-docs :exec-args {:name "coffi" - :version "v0.5.357" - :description "A Foreign Function Interface in Clojure for JDK 18." + :version "v0.6.409" + :description "A Foreign Function Interface in Clojure for JDK 19." :source-paths ["src/clj"] :output-path "docs" :source-uri "https://github.com/IGJoshua/coffi/blob/{git-commit}/{filepath}#L{line}" diff --git a/docs/coffi.ffi.html b/docs/coffi.ffi.html index ac3e8c1..eef6ee7 100644 --- a/docs/coffi.ffi.html +++ b/docs/coffi.ffi.html @@ -1,19 +1,20 @@ -coffi.ffi documentation

Project

Namespaces

Public Vars

coffi.ffi

Functions for creating handles to native functions and loading native libraries.

cfn

(cfn symbol args ret)

Constructs a Clojure function to call the native function referenced by symbol.

+coffi.ffi documentation

Project

Namespaces

Public Vars

coffi.ffi

Functions for creating handles to native functions and loading native libraries.

cfn

(cfn symbol args ret)

Constructs a Clojure function to call the native function referenced by symbol.

The function returned will serialize any passed arguments into the args types, and deserialize the return to the ret type.

-

If your args and ret are constants, then it is more efficient to call make-downcall followed by make-serde-wrapper because the latter has an inline definition which will result in less overhead from serdes.

view source

const

(const symbol-or-addr type)

Gets the value of a constant stored in symbol-or-addr.

view source

defcfn

macro

(defcfn name docstring? attr-map? symbol arg-types ret-type)(defcfn name docstring? attr-map? symbol arg-types ret-type native-fn & fn-tail)

Defines a Clojure function which maps to a native function.

+

If your args and ret are constants, then it is more efficient to call make-downcall followed by make-serde-wrapper because the latter has an inline definition which will result in less overhead from serdes.

view source

const

(const symbol-or-addr type)

Gets the value of a constant stored in symbol-or-addr.

view source

defcfn

macro

(defcfn name docstring? attr-map? symbol arg-types ret-type)(defcfn name docstring? attr-map? symbol arg-types ret-type native-fn & fn-tail)

Defines a Clojure function which maps to a native function.

name is the symbol naming the resulting var. symbol is a symbol or string naming the library symbol to link against. arg-types is a vector of qualified keywords representing the argument types. ret-type is a single qualified keyword representing the return type. fn-tail is the body of the function (potentially with multiple arities) which wraps the native one. Inside the function, native-fn is bound to a function that will serialize its arguments, call the native function, and deserialize its return type. If any body is present, you must call this function in order to call the native code.

If no fn-tail is provided, then the resulting function will simply serialize the arguments according to arg-types, call the native function, and deserialize the return value.

The number of args in the fn-tail need not match the number of arg-types for the native function. It need only call the native wrapper function with the correct arguments.

-

See serialize, deserialize, make-downcall.

view source

find-symbol

(find-symbol sym)

Gets the NativeSymbol of a symbol from the loaded libraries.

view source

freset!

(freset! static-var newval)

Sets the value of static-var to newval, running it through serialize.

view source

fswap!

(fswap! static-var f & args)

Non-atomically runs the function f over the value stored in static-var.

-

The value is deserialized before passing it to f, and serialized before putting the value into static-var.

view source

load-library

(load-library path)

Loads the library at path.

view source

load-system-library

(load-system-library libname)

Loads the library named libname from the system’s load path.

view source

make-downcall

(make-downcall symbol-or-addr args ret)

Constructs a downcall function reference to symbol-or-addr with the given args and ret types.

+

See serialize, deserialize, make-downcall.

view source

defconst

macro

(defconst symbol docstring? symbol-or-addr type)

Defines a var named by symbol to be the value of the given type from symbol-or-addr.

view source

defvar

macro

(defvar symbol docstring? symbol-or-addr type)

Defines a var named by symbol to be a reference to the native memory from symbol-or-addr.

view source

ensure-symbol

(ensure-symbol symbol-or-addr)

Returns the argument if it is a MemorySegment, otherwise calls find-symbol on it.

view source

find-symbol

(find-symbol sym)

Gets the MemorySegment of a symbol from the loaded libraries.

view source

freset!

(freset! static-var newval)

Sets the value of static-var to newval, running it through serialize.

view source

fswap!

(fswap! static-var f & args)

Non-atomically runs the function f over the value stored in static-var.

+

The value is deserialized before passing it to f, and serialized before putting the value into static-var.

view source

load-library

(load-library path)

Loads the library at path.

view source

load-system-library

(load-system-library libname)

Loads the library named libname from the system’s load path.

view source

make-downcall

(make-downcall symbol-or-addr args ret)

Constructs a downcall function reference to symbol-or-addr with the given args and ret types.

The function returned takes only arguments whose types match exactly the java-layout for that type, and returns an argument with exactly the java-layout of the ret type. This function will perform no serialization or deserialization of arguments or the return type.

-

If the ret type is non-primitive, then the returned function will take a first argument of a SegmentAllocator.

view source

make-serde-varargs-wrapper

(make-serde-varargs-wrapper varargs-factory required-args ret-type)

Constructs a wrapper function for the varargs-factory which produces functions that serialize the arguments and deserialize the return value.

view source

make-serde-wrapper

(make-serde-wrapper downcall arg-types ret-type)

Constructs a wrapper function for the downcall which serializes the arguments and deserializes the return value.

view source

make-varargs-factory

(make-varargs-factory symbol required-args ret)

Returns a function for constructing downcalls with additional types for arguments.

+

If the ret type is non-primitive, then the returned function will take a first argument of a SegmentAllocator.

view source

make-serde-varargs-wrapper

(make-serde-varargs-wrapper varargs-factory required-args ret-type)

Constructs a wrapper function for the varargs-factory which produces functions that serialize the arguments and deserialize the return value.

view source

make-serde-wrapper

(make-serde-wrapper downcall arg-types ret-type)

Constructs a wrapper function for the downcall which serializes the arguments and deserializes the return value.

view source

make-varargs-factory

(make-varargs-factory symbol required-args ret)

Returns a function for constructing downcalls with additional types for arguments.

The required-args are the types of the first arguments passed to the downcall handle, and the values passed to the returned function are only the varargs types.

The returned function is memoized, so that only one downcall function will be generated per combination of argument types.

-

See make-downcall.

view source

reify-libspec

(reify-libspec libspec)

Loads all the symbols specified in the libspec.

-

The value of each key of the passed map is transformed as by reify-symbolspec.

view source

reify-symbolspec

multimethod

Takes a spec for a symbol reference and returns a live value for that type.

view source

static-variable

(static-variable symbol-or-addr type)

Constructs a reference to a mutable value stored in symbol-or-addr.

-

The returned value can be dereferenced, and has metadata, and the address of the value can be queried with address-of.

-

See freset!, fswap!.

view source

vacfn-factory

(vacfn-factory symbol required-args ret)

Constructs a varargs factory to call the native function referenced by symbol.

-

The function returned takes any number of type arguments and returns a specialized Clojure function for calling the native function with those arguments.

view source
\ No newline at end of file +

See make-downcall.

view source

reify-libspec

(reify-libspec libspec)

Loads all the symbols specified in the libspec.

+

The value of each key of the passed map is transformed as by reify-symbolspec.

view source

reify-symbolspec

multimethod

Takes a spec for a symbol reference and returns a live value for that type.

view source

static-variable

(static-variable symbol-or-addr type)

Constructs a reference to a mutable value stored in symbol-or-addr.

+

The returned value can be dereferenced, and has metadata.

+

See freset!, fswap!.

view source

static-variable-segment

(static-variable-segment static-var)

Gets the backing MemorySegment from static-var.

+

This is primarily useful when you need to pass the static variable’s address to a native function which takes an Addressable.

view source

vacfn-factory

(vacfn-factory symbol required-args ret)

Constructs a varargs factory to call the native function referenced by symbol.

+

The function returned takes any number of type arguments and returns a specialized Clojure function for calling the native function with those arguments.

view source
\ No newline at end of file diff --git a/docs/coffi.layout.html b/docs/coffi.layout.html index 0d699fe..2b49846 100644 --- a/docs/coffi.layout.html +++ b/docs/coffi.layout.html @@ -1,4 +1,4 @@ -coffi.layout documentation

Project

Namespaces

Public Vars

coffi.layout

Functions for adjusting the layout of structs.

with-c-layout

(with-c-layout struct-spec)

Forces a struct specification to C layout rules.

-

This will add padding fields between fields to match C alignment requirements.

view source
\ No newline at end of file +coffi.layout documentation

Project

Namespaces

Public Vars

coffi.layout

Functions for adjusting the layout of structs.

with-c-layout

(with-c-layout struct-spec)

Forces a struct specification to C layout rules.

+

This will add padding fields between fields to match C alignment requirements.

view source
\ No newline at end of file diff --git a/docs/coffi.mem.html b/docs/coffi.mem.html index 0378250..a2c8534 100644 --- a/docs/coffi.mem.html +++ b/docs/coffi.mem.html @@ -1,49 +1,55 @@ -coffi.mem documentation

Project

Namespaces

Public Vars

coffi.mem

Functions for managing native allocations, resource scopes, and (de)serialization.

+coffi.mem documentation

Project

Namespaces

Public Vars

coffi.mem

Functions for managing native allocations, memory sessions, and (de)serialization.

For any new type to be implemented, three multimethods must be overriden, but which three depends on the native representation of the type.

If the native representation of the type is a primitive (whether or not other data beyond the primitive is associated with it, as e.g. a pointer), then primitive-type must be overriden to return which primitive type it is serialized as, then serialize* and deserialize* should be overriden.

If the native representation of the type is a composite type, like a union, struct, or array, then c-layout must be overriden to return the native layout of the type, and serialize-into and deserialize-from should be overriden to allow marshaling values of the type into and out of memory segments.

-

When writing code that manipulates a segment, it’s best practice to use with-acquired on the segment-scope in order to ensure it won’t be released during its manipulation.

add-close-action!

(add-close-action! scope action)

Adds a 0-arity function to be run when the scope closes.

view source

address-of

(address-of addressable)

Gets the address of a given segment.

-

This value can be used as an argument to functions which take a pointer.

view source

address?

(address? addr)

Checks if an object is a memory address.

-

nil is considered an address.

view source

align-of

(align-of type)

The alignment in bytes of the given type.

view source

alloc

(alloc size)(alloc size scope)

Allocates size bytes.

-

If a scope is provided, the allocation will be reclaimed when it is closed.

view source

alloc-instance

(alloc-instance type)(alloc-instance type scope)

Allocates a memory segment for the given type.

view source

alloc-with

(alloc-with allocator size)(alloc-with allocator size alignment)

Allocates size bytes using the allocator.

view source

as-segment

(as-segment address size)(as-segment address size scope)

Dereferences an address into a memory segment associated with the scope.

view source

big-endian

The big-endian ByteOrder.

-

See little-endian, native-endian.

view source

byte-layout

The MemoryLayout for a byte in native-endian ByteOrder.

view source

c-layout

multimethod

Gets the layout object for a given type.

+

When writing code that manipulates a segment, it’s best practice to use with-acquired on the segment-session in order to ensure it won’t be released during its manipulation.

add-close-action!

(add-close-action! session action)

Adds a 0-arity function to be run when the session closes.

view source

address-of

(address-of addressable)

Gets the address of a given segment.

+

This value can be used as an argument to functions which take a pointer.

view source

address?

(address? addr)

Checks if an object is a memory address.

+

nil is considered an address.

view source

align-of

(align-of type)

The alignment in bytes of the given type.

view source

alloc

(alloc size)(alloc size session)(alloc size alignment session)

Allocates size bytes.

+

If a session is provided, the allocation will be reclaimed when it is closed.

view source

alloc-instance

(alloc-instance type)(alloc-instance type session)

Allocates a memory segment for the given type.

view source

alloc-with

(alloc-with allocator size)(alloc-with allocator size alignment)

Allocates size bytes using the allocator.

view source

as-segment

(as-segment address size)(as-segment address size session)

Dereferences an address into a memory segment associated with the session.

view source

big-endian

The big-endian ByteOrder.

+

See little-endian, native-endian.

view source

byte-layout

The MemoryLayout for a byte in native-endian ByteOrder.

view source

c-layout

multimethod

Gets the layout object for a given type.

If a type is primitive it will return the appropriate primitive layout (see c-prim-layout).

-

Otherwise, it should return a GroupLayout for the given type.

view source

char-layout

The MemoryLayout for a c-sized char in native-endian ByteOrder.

view source

clone-segment

(clone-segment segment)(clone-segment segment scope)

Clones the content of segment into a new segment of the same size.

view source

connected-scope

(connected-scope)

Constructs a new scope to reclaim all connected resources at once.

+

Otherwise, it should return a GroupLayout for the given type.

view source

char-layout

The MemoryLayout for a c-sized char in native-endian ByteOrder.

view source

clone-segment

(clone-segment segment)(clone-segment segment session)

Clones the content of segment into a new segment of the same size.

view source

connected-scope

deprecated

(connected-scope)

Constructs a new scope to reclaim all connected resources at once.

The scope may be shared across threads, and all resources created with it will be cleaned up at the same time, when all references have been collected.

-

This type of scope cannot be closed, and therefore should not be created in a with-open clause.

view source

copy-segment

(copy-segment dest src)

Copies the content to dest from src.

-

Returns dest.

view source

defalias

macro

(defalias new-type aliased-type)

Defines a type alias from new-type to aliased-type.

-

This creates needed serialization and deserialization implementations for the aliased type.

view source

deserialize

(deserialize obj type)

Deserializes an arbitrary type.

-

For types which have a primitive representation, this deserializes the primitive representation. For types which do not, this deserializes out of a segment.

view source

deserialize*

multimethod

Deserializes a primitive object into a Clojure data structure.

-

This is intended for use with types that are returned as a primitive but which need additional processing before they can be returned.

view source

deserialize-from

multimethod

Deserializes the given segment into a Clojure data structure.

+

This type of scope cannot be closed, and therefore should not be created in a with-open clause.

view source

connected-session

(connected-session)

Constructs a new memory session to reclaim all connected resources at once.

+

The session may be shared across threads, and all resources created with it will be cleaned up at the same time, when all references have been collected.

+

This type of session cannot be closed, and therefore should not be created in a with-open clause.

view source

copy-segment

(copy-segment dest src)

Copies the content to dest from src.

+

Returns dest.

view source

defalias

macro

(defalias new-type aliased-type)

Defines a type alias from new-type to aliased-type.

+

This creates needed serialization and deserialization implementations for the aliased type.

view source

deserialize

(deserialize obj type)

Deserializes an arbitrary type.

+

For types which have a primitive representation, this deserializes the primitive representation. For types which do not, this deserializes out of a segment.

view source

deserialize*

multimethod

Deserializes a primitive object into a Clojure data structure.

+

This is intended for use with types that are returned as a primitive but which need additional processing before they can be returned.

view source

deserialize-from

multimethod

Deserializes the given segment into a Clojure data structure.

For types that serialize to primitives, a default implementation will deserialize the primitive before calling deserialize*.

-

Implementations of this should be inside a with-acquired block for the the segment’s scope if they perform multiple memory operations.

view source

double-alignment

The alignment in bytes of a c-sized double.

view source

double-layout

The MemoryLayout for a c-sized double in native-endian ByteOrder.

view source

double-size

The size in bytes of a c-sized double.

view source

float-alignment

The alignment in bytes of a c-sized float.

view source

float-layout

The MemoryLayout for a c-sized float in native-endian ByteOrder.

view source

float-size

The size in bytes of a c-sized float.

view source

global-scope

(global-scope)

Constructs the global scope, which will never reclaim its resources.

-

This scope may be shared across threads, but is intended mainly in cases where memory is allocated with alloc but is either never freed or whose management is relinquished to a native library, such as when returned from a callback.

view source

int-alignment

The alignment in bytes of a c-sized int.

view source

int-layout

The MemoryLayout for a c-sized int in native-endian ByteOrder.

view source

int-size

The size in bytes of a c-sized int.

view source

java-layout

(java-layout type)

Gets the Java class to an argument of this type for a method handle.

-

If a type serializes to a primitive it returns return a Java primitive type. Otherwise, it returns MemorySegment.

view source

java-prim-layout

Map of primitive type names to the Java types for a method handle.

view source

little-endian

The little-endian ByteOrder.

-

See big-endian, native-endian

view source

long-alignment

The alignment in bytes of a c-sized long.

view source

long-layout

The MemoryLayout for a c-sized long in native-endian ByteOrder.

view source

long-size

The size in bytes of a c-sized long.

view source

native-endian

The ByteOrder for the native endianness of the current hardware.

-

See big-endian, little-endian.

view source

null?

(null? addr)

Checks if a memory address is null.

view source

pointer-alignment

The alignment in bytes of a c-sized pointer.

view source

pointer-layout

The MemoryLayout for a native pointer in native-endian ByteOrder.

view source

pointer-size

The size in bytes of a c-sized pointer.

view source

primitive-type

multimethod

Gets the primitive type that is used to pass as an argument for the type.

+

Implementations of this should be inside a with-acquired block for the the segment’s session if they perform multiple memory operations.

view source

double-alignment

The alignment in bytes of a c-sized double.

view source

double-layout

The MemoryLayout for a c-sized double in native-endian ByteOrder.

view source

double-size

The size in bytes of a c-sized double.

view source

float-alignment

The alignment in bytes of a c-sized float.

view source

float-layout

The MemoryLayout for a c-sized float in native-endian ByteOrder.

view source

float-size

The size in bytes of a c-sized float.

view source

global-scope

deprecated

(global-scope)

Constructs the global scope, which will never reclaim its resources.

+

This scope may be shared across threads, but is intended mainly in cases where memory is allocated with alloc but is either never freed or whose management is relinquished to a native library, such as when returned from a callback.

view source

global-session

(global-session)

Constructs the global session, which will never reclaim its resources.

+

This session may be shared across threads, but is intended mainly in cases where memory is allocated with alloc but is either never freed or whose management is relinquished to a native library, such as when returned from a callback.

view source

int-alignment

The alignment in bytes of a c-sized int.

view source

int-layout

The MemoryLayout for a c-sized int in native-endian ByteOrder.

view source

int-size

The size in bytes of a c-sized int.

view source

java-layout

(java-layout type)

Gets the Java class to an argument of this type for a method handle.

+

If a type serializes to a primitive it returns return a Java primitive type. Otherwise, it returns MemorySegment.

view source

java-prim-layout

Map of primitive type names to the Java types for a method handle.

view source

little-endian

The little-endian ByteOrder.

+

See big-endian, native-endian

view source

long-alignment

The alignment in bytes of a c-sized long.

view source

long-layout

The MemoryLayout for a c-sized long in native-endian ByteOrder.

view source

long-size

The size in bytes of a c-sized long.

view source

native-endian

The ByteOrder for the native endianness of the current hardware.

+

See big-endian, little-endian.

view source

null?

(null? addr)

Checks if a memory address is null.

view source

pointer-alignment

The alignment in bytes of a c-sized pointer.

view source

pointer-layout

The MemoryLayout for a native pointer in native-endian ByteOrder.

view source

pointer-size

The size in bytes of a c-sized pointer.

view source

primitive-type

multimethod

Gets the primitive type that is used to pass as an argument for the type.

This is for objects which are passed to native functions as primitive types, but which need additional logic to be performed during serialization and deserialization.

Implementations of this method should take into account that type arguments may not always be evaluated before passing to this function.

-

Returns nil for any type which does not have a primitive representation.

view source

primitive-types

A set of all primitive types.

view source

primitive?

(primitive? type)

A predicate to determine if a given type is primitive.

view source

read-address

(read-address segment)(read-address segment offset)

Reads a MemoryAddress from the segment, at an optional offset.

view source

read-byte

(read-byte segment)(read-byte segment offset)

Reads a byte from the segment, at an optional offset.

view source

read-char

(read-char segment)(read-char segment offset)

Reads a char from the segment, at an optional offset.

view source

read-double

(read-double segment)(read-double segment offset)(read-double segment offset byte-order)

Reads a double from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

read-float

(read-float segment)(read-float segment offset)(read-float segment offset byte-order)

Reads a float from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

read-int

(read-int segment)(read-int segment offset)(read-int segment offset byte-order)

Reads a int from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

read-long

(read-long segment)(read-long segment offset)(read-long segment offset byte-order)

Reads a long from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

read-short

(read-short segment)(read-short segment offset)(read-short segment offset byte-order)

Reads a short from the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

scope-allocator

(scope-allocator scope)

Constructs a segment allocator from the given scope.

-

This is primarily used when working with unwrapped downcall functions. When a downcall function returns a non-primitive type, it must be provided with an allocator.

view source

segment-scope

(segment-scope segment)

Gets the scope used to construct the segment.

view source

seq-of

(seq-of type segment)

Constructs a lazy sequence of type elements deserialized from segment.

view source

serialize

(serialize obj type)(serialize obj type scope)

Serializes an arbitrary type.

-

For types which have a primitive representation, this serializes into that representation. For types which do not, it allocates a new segment and serializes into that.

view source

serialize*

multimethod

Constructs a serialized version of the obj and returns it.

-

Any new allocations made during the serialization should be tied to the given scope, except in extenuating circumstances.

-

This method should only be implemented for types that serialize to primitives.

view source

serialize-into

multimethod

Writes a serialized version of the obj to the given segment.

-

Any new allocations made during the serialization should be tied to the given scope, except in extenuating circumstances.

+

Returns nil for any type which does not have a primitive representation.

view source

primitive-types

A set of all primitive types.

view source

primitive?

(primitive? type)

A predicate to determine if a given type is primitive.

view source

read-address

(read-address segment)(read-address segment offset)

Reads a MemoryAddress from the segment, at an optional offset.

view source

read-byte

(read-byte segment)(read-byte segment offset)

Reads a byte from the segment, at an optional offset.

view source

read-char

(read-char segment)(read-char segment offset)

Reads a char from the segment, at an optional offset.

view source

read-double

(read-double segment)(read-double segment offset)(read-double segment offset byte-order)

Reads a double from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

read-float

(read-float segment)(read-float segment offset)(read-float segment offset byte-order)

Reads a float from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

read-int

(read-int segment)(read-int segment offset)(read-int segment offset byte-order)

Reads a int from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

read-long

(read-long segment)(read-long segment offset)(read-long segment offset byte-order)

Reads a long from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

read-short

(read-short segment)(read-short segment offset)(read-short segment offset byte-order)

Reads a short from the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

scope-allocator

deprecated

(scope-allocator scope)

Constructs a segment allocator from the given scope.

+

This is primarily used when working with unwrapped downcall functions. When a downcall function returns a non-primitive type, it must be provided with an allocator.

view source

segment-scope

deprecated

(segment-scope segment)

Gets the scope used to construct the segment.

view source

segment-session

(segment-session segment)

Gets the memory session used to construct the segment.

view source

seq-of

(seq-of type segment)

Constructs a lazy sequence of type elements deserialized from segment.

view source

serialize

(serialize obj type)(serialize obj type session)

Serializes an arbitrary type.

+

For types which have a primitive representation, this serializes into that representation. For types which do not, it allocates a new segment and serializes into that.

view source

serialize*

multimethod

Constructs a serialized version of the obj and returns it.

+

Any new allocations made during the serialization should be tied to the given session, except in extenuating circumstances.

+

This method should only be implemented for types that serialize to primitives.

view source

serialize-into

multimethod

Writes a serialized version of the obj to the given segment.

+

Any new allocations made during the serialization should be tied to the given session, except in extenuating circumstances.

This method should be implemented for any type which does not override c-layout.

For any other type, this will serialize it as serialize* before writing the result value into the segment.

-

Implementations of this should be inside a with-acquired block for the scope if they perform multiple memory operations.

view source

shared-scope

(shared-scope)

Constructs a new shared scope.

-

This scope can be shared across threads and memory allocated in it will only be cleaned up once every thread accessing the scope closes it.

view source

short-alignment

The alignment in bytes of a c-sized short.

view source

short-layout

The MemoryLayout for a c-sized short in native-endian ByteOrder.

view source

short-size

The size in bytes of a c-sized short.

view source

size-of

(size-of type)

The size in bytes of the given type.

view source

slice

(slice segment offset)(slice segment offset size)

Get a slice over the segment with the given offset.

view source

slice-into

(slice-into address segment)(slice-into address segment size)

Get a slice into the segment starting at the address.

view source

slice-segments

(slice-segments segment size)

Constructs a lazy seq of size-length memory segments, sliced from segment.

view source

stack-scope

(stack-scope)

Constructs a new scope for use only in this thread.

-

The memory allocated within this scope is cheap to allocate, like a native stack.

view source

with-acquired

macro

(with-acquired scopes & body)

Acquires one or more scopes until the body completes.

-

This is only necessary to do on shared scopes, however if you are operating on an arbitrary passed scope, it is best practice to wrap code that interacts with it wrapped in this.

view source

with-offset

(with-offset address offset)

Get a new address offset from the old address.

view source

write-address

(write-address segment value)(write-address segment offset value)

Writes a MemoryAddress to the segment, at an optional offset.

view source

write-byte

(write-byte segment value)(write-byte segment offset value)

Writes a byte to the segment, at an optional offset.

view source

write-char

(write-char segment value)(write-char segment offset value)

Writes a char to the segment, at an optional offset.

view source

write-double

(write-double segment value)(write-double segment offset value)(write-double segment offset byte-order value)

Writes a double to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

write-float

(write-float segment value)(write-float segment offset value)(write-float segment offset byte-order value)

Writes a float to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

write-int

(write-int segment value)(write-int segment offset value)(write-int segment offset byte-order value)

Writes a int to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

write-long

(write-long segment value)(write-long segment offset value)(write-long segment offset byte-order value)

Writes a long to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source

write-short

(write-short segment value)(write-short segment offset value)(write-short segment offset byte-order value)

Writes a short to the segment, at an optional offset.

-

If byte-order is not provided, it defaults to native-endian.

view source
\ No newline at end of file +

Implementations of this should be inside a with-acquired block for the session if they perform multiple memory operations.

view source

session-allocator

(session-allocator session)

Constructs a segment allocator from the given session.

+

This is primarily used when working with unwrapped downcall functions. When a downcall function returns a non-primitive type, it must be provided with an allocator.

view source

shared-scope

deprecated

(shared-scope)

Constructs a new shared scope.

+

This scope can be shared across threads and memory allocated in it will only be cleaned up once every thread accessing the scope closes it.

view source

shared-session

(shared-session)(shared-session cleaner)

Constructs a new shared memory session.

+

This session can be shared across threads and memory allocated in it will only be cleaned up once every thread accessing the session closes it.

view source

short-alignment

The alignment in bytes of a c-sized short.

view source

short-layout

The MemoryLayout for a c-sized short in native-endian ByteOrder.

view source

short-size

The size in bytes of a c-sized short.

view source

size-of

(size-of type)

The size in bytes of the given type.

view source

slice

(slice segment offset)(slice segment offset size)

Get a slice over the segment with the given offset.

view source

slice-into

(slice-into address segment)(slice-into address segment size)

Get a slice into the segment starting at the address.

view source

slice-segments

(slice-segments segment size)

Constructs a lazy seq of size-length memory segments, sliced from segment.

view source

stack-scope

deprecated

(stack-scope)

Constructs a new scope for use only in this thread.

+

The memory allocated within this scope is cheap to allocate, like a native stack.

view source

stack-session

(stack-session)(stack-session cleaner)

Constructs a new session for use only in this thread.

+

The memory allocated within this session is cheap to allocate, like a native stack.

view source

with-acquired

macro

(with-acquired sessions & body)

Acquires one or more sessions until the body completes.

+

This is only necessary to do on shared sessions, however if you are operating on an arbitrary passed session, it is best practice to wrap code that interacts with it wrapped in this.

view source

with-offset

(with-offset address offset)

Get a new address offset from the old address.

view source

write-address

(write-address segment value)(write-address segment offset value)

Writes a MemoryAddress to the segment, at an optional offset.

view source

write-byte

(write-byte segment value)(write-byte segment offset value)

Writes a byte to the segment, at an optional offset.

view source

write-char

(write-char segment value)(write-char segment offset value)

Writes a char to the segment, at an optional offset.

view source

write-double

(write-double segment value)(write-double segment offset value)(write-double segment offset byte-order value)

Writes a double to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

write-float

(write-float segment value)(write-float segment offset value)(write-float segment offset byte-order value)

Writes a float to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

write-int

(write-int segment value)(write-int segment offset value)(write-int segment offset byte-order value)

Writes a int to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

write-long

(write-long segment value)(write-long segment offset value)(write-long segment offset byte-order value)

Writes a long to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source

write-short

(write-short segment value)(write-short segment offset value)(write-short segment offset byte-order value)

Writes a short to the segment, at an optional offset.

+

If byte-order is not provided, it defaults to native-endian.

view source
\ No newline at end of file diff --git a/docs/index.html b/docs/index.html index ea9780b..50a6e50 100644 --- a/docs/index.html +++ b/docs/index.html @@ -1,3 +1,3 @@ -coffi v0.5.357

Project

Namespaces

coffi v0.5.357

A Foreign Function Interface in Clojure for JDK 18.

Namespaces

coffi.ffi

Functions for creating handles to native functions and loading native libraries.

Public variables and functions:

coffi.layout

Functions for adjusting the layout of structs.

Public variables and functions:

coffi.mem

Functions for managing native allocations, resource scopes, and (de)serialization.

Public variables and functions:

\ No newline at end of file +coffi v0.6.409

Project

Namespaces

coffi v0.6.409

A Foreign Function Interface in Clojure for JDK 19.

Namespaces

coffi.ffi

Functions for creating handles to native functions and loading native libraries.

Public variables and functions:

coffi.layout

Functions for adjusting the layout of structs.

Public variables and functions:

coffi.mem

Functions for managing native allocations, memory sessions, and (de)serialization.

Public variables and functions:

\ No newline at end of file