* Preliminary ReduceProd Support
* Add comma to keep formatter happy
* Give test results a 0.001 tolerance to account for floating-point multiplication
* Reformat assersions
* Correctly mark panic conditions in op_configuration
* Move distribution to module
* Add new TensorData with serialization support
* Implement display and from for TensorData
* Add missing Cargo.lock
* Add missing bytemuck feature
* Add zeros, ones, full and random TensorData methods
* Refactor Data -> TensorData usage
* Fix tests
Since TensorData is not generic over the element type anymore no type inference can be done by the compiler. We must explicitly cast the expected results to the expected backend type.
* Remove commented line
* Fix import
* Add record-backward-compat
* Remove dim const generic from TensorData
* Support NestedValue de/serialization with TensorData
* Fix burn-jit tests
* Remove eprinln
* Refactor onnx import to use TensorData
* Fix tch from_data
* Fix nested value serialization for u8
* Fix missing import
* Fix reduce min onnx test
* Fix deprecated attribute
* Remove shape getter
* Remove strict assert in tests
* Add tensor data as_bytes
* Add tensor check for rank mismatch
* Fix typo (dimensions plural)
* Fix error message
* Update book examples with from_data and fix Display impl for TensorData
* Add deprecation note
* Element already implements One
* Add element module
* Add our own traits for Zero, One and ToPrimitive to support bool Element
* Fix typo
* Add basic tests for ToPrimitive with expected values
* The most important change of all
* Remove One + Zero identities
* Move zero/one outside mapv + refactor ToPrimitive -> ToElement trait
* Add num-traits to NOTICES.md
* Updated documentation for unfold4d
Added links between the struct and the config. Added a link to the related burn_tensor function in the documentation for the forward function.
* Changing nn relu module documentation to functional api
Removing the formula for relu from the module API to the functional API,
citing a paper relevant to relu
and mentionning the functional API in the module API
* Linking gelu module API documentation to functional API documentation
* Linear module : adding documentation
Adding documentation to the Linear module
mentionning that LinearConfig struct
should be used when creating a Linear Layer
Also adding links to the documentation that points people toward
the right path
* Updated documentation for dropout
Added links between the struct and the config. Added a link to the struct in the forward function for more info.
* embedding + swiglu
* RotaryEncodying : adding documentation
Adding documentation stating the RotaryEncoding should be created using a RotaryEncodingConfig
* prelu: adding documentation
Adding documentation to the prelu module:
- Linking forward function documentation to the functional API
- Citing the first paper to mention prelu
- Adding documentation saying that prelu layer should be created using PReluConfig
* pos_encoding: adding documentation
* Updated documentation for mha
Added links for more info. Added shape info at some places.
* docs: Add documentation for Gru module
Provide documentation for the Gru module, including its configuration and usage. Include a link to the paper that introduced the Gated Recurrent Unit (GRU) and specify that the module should be created using GruConfig. Also, mention that the forward function returns a state tensor with specific dimensions.
* burn-core-nn-transformers: adding documentation
Adding documentation:
- Says to use config to create the layers
- Add mathematical formula to the pwff forward pass
- Add citation in the pwff to the "Attention is all you need" paper
* Updated documentation: ConvTranspose1d and ConvTranspose2d
* docs: Add documentation for Lstm and BiLstm modules
Provide documentation for the Lstm and BiLstm modules, including their configurations and usage. Include links to the papers that introduced Long Short-Term Memory (LSTM) and Bidirectional LSTM. Specify that the modules should be created using LstmConfig and BiLstmConfig respectively.
* docs: Update documentation for ConvTranspose1d and ConvTranspose2d modules
* loss: Adding documenntation to the loss layers
Adding documentation stating to use the config to create the layer
* chore: Refactor Conv1d module imports and update documentation
* docs: Add documentation for AdaptiveAvgPool1d and AdaptiveAvgPool2d modules
Added references to the burn_tensor associated functions. Added links between the struct and the config.
* Refactor Conv1d module imports and update documentation
* chore: Refactor Conv2d module imports and update documentation
* Add documentation for AvgPool1d and AvgPool2d modules
Added references to the burn_tensor associated functions. Added links between the struct and the config.
* Add documentation for MaxPool1d and MaxPool2d modules
Added references to the burn_tensor associated functions. Added links between the struct and the config.
* Add documentation for leaky_relu and removed Config generic
Added references to the burn_tensor associated functions. Added links between the struct and the config. Removed the backend generic from the config since it's not needed (might be a breaking change).
* refactor: Update BatchNormConfig initialization and add documentation.
* Added link to config in embedding struct documentation
* refactor: Update GroupNormConfig initialization and add documentation
* refactor: Update InstanceNormConfig initialization and add documentation
* feat: Update LayerNormConfig initialization and add documentation
* refactor: Update RmsNormConfig initialization and add documentation
* fixed: removed #derive accidentally
* Added missing backticks in pools' shapes
* Format nn doc
* Make config fields public in nn modules
* Update import statements in nn modules
Changed burn_tensor imports to crate::tensor
* Update import statements in nn modules' tests
Changed burn_tensor imports to crate::tensor
* breaking change refactor: Update GroupNormConfig and InstanceNormConfig initialization
* Make SwiGlu fields public
* grammar
* slashes
* input tensors grouping
* copy-pasta mistake
* a not an >:I
* Capitalization
* better desc
* math 'n ticks
* group_norm functional implementation
* removed the ... struct
* decoder typo
* fmt
* referring to private fn in docs
---------
Co-authored-by: Thierry Cantin-Demers <piertcd@gmail.com>
Co-authored-by: mepatrick73 <pameu17@ulaval.ca>
* draft for alternative burn import design
* passes onnx test, fails to build example
* pushing to test example on main
* fixed the issue with the example
* passes the test now
* spring cleaning and minor code changes
* removed pub visibility from most graph_data fields and functions
* comment fixes
* went ahead and removed the constant check for now
* removed unused function arg
* Add a feature to initialize from an existing wgpu adapter/device/queue
This is useful when interacting with other wgpu applications (eg. displaying a burn tensor as a texture in egui). The existing devices are keyed by the wgpu Device ID. Alternatively they could be keyed per adapter which would be more inline with other burn WgpuDevice's (one per adapter), but also there's no real inherent reason to.
This also involves making Queue into an Arc. Alternatively, this could give up ownership of the queue, but it's helpful to be able to synchronize burn operations and custom wgpu operations.
* struct support (receive, use and modify fields)
* support struct with generics
* expect instead of unwrap
* fmt
* rename struc
* fmt
* Clippy
* Fix launcher
* Support creating private cube type without generics
* Cleanup
* generics support
* clippy
* minor
* fmt
---------
Co-authored-by: nathaniel <nathaniel.simard.42@gmail.com>
* Move and redirect GatherElements to new folders/nodes
* Create PyTorch script for gather
* Add onnx file for gather
* Add a gather test to onnx_tests
* Update gather.rs to use select
* Rename codegen test
* Update gather and gather_elements conversion functions
* Validate rank of input node and update output
* Add check for Gather
* pre-test
* implementing argmax for burn-import from onnx
* tidying
* fixing return types and tests
* addressing feedback
* only warn when select_last_index!=0
* Ben WIP
* Compile burn-jit
* WGPU works
* Remove old code
* move language cube stuff
* cleaning up
* some import reworking
* remove cube reexport
* template feature flag in cube
* ci
---------
Co-authored-by: nathaniel <nathaniel.simard.42@gmail.com>
* #1747
Upgrade Rust dependencies
* Revert upgrade for tch
The update of tch on windows gives an error:
INTEL MKL ERROR: The specified module could not be found. mkl_vml_avx2.1.dll.
Intel MKL FATAL ERROR: cannot load mkl_vml_avx2.1.dll or mkl_vml_def.1.dll.
* Keep only .cargo/config.toml file which works with rust > 1.75
---------
Co-authored-by: Sylvain Benner <sylvain@benner.online>
* resolve conflict
* move `gate_product` to `GateController`
* BiLstm needs to use its own initializer when init
* resolve conflicts
* add some comments
* improve doc
* correct the description of GateController
* fix fmt
* add `LstmState`
* add test for state
* set batch 2 in bilstm test
* resolve conflict
* fix
* fix doc
* change the batch size back to 1
* change the batch size back to 1
* modify docstring; delete dead comment
* Move HandlerContainer and Tensor Ops description to burn-tensor
Move HandleContainer and Tensor operations descriptions to burn-tensor crate.
Removed the FusionDevice and replaced it with a DeviceOps trait bound to Backend::Device.
For now added modules to burn-tensor are excluded from no-std as they rely on Arc.
* [burn-tensor] Flatten module hierarchy for tensor representation
+ Add new repr feature to cargo file.
* Remove prefix on dosctring
* [burn-fusion] Require default features of burn-tensor
* Add where onnx op support
* Add broadcasting support
* Remove broadcasting limitation comment
* Fix broadcasting in mask where
* Forgot to reflect changes in codegen test
* Fix clippy
* Mul onnx op already supported
* Add matmul onnx op checks and tests
* Add missing eq derives
* Change supscript symbol
* Remove dead code
* Add support for matmul broadcast
* No more broadcasting restrictions
* Add results comment for mm, mv and vm
* Add remainder_scalar op to numeric trait and associated int/float functions
* Update burn-tch crate
* Update ndarray crate
* Update jit crate
* Update candle crate
* Update fusion crate
* Update autodiff crate
* Forgot float.rs for fusion
* Add burn-tensor tests
* Redirect to the pre-existing modulus op
* Fix sign
* Remove mut from burn-tch
* Use sign trick to make wgpu backend work
* Add more unit tests in to cover bases
* Naming fix for burn-fusion
* Update tests w/PyTorch link
* Use different WGSL instructions for remainder
* Redirect to remainder Operator instead of modulo
* Revert Modulo in instruction.rs
* Update links to latest commit off main
* Some pedantry
* Update links and add jit
* Update instructions for burn-jit and wgpu
* Updated import section with more recent links
* Some grammar/typo/styling fixes
* Code added to burn-wgpu too
* Add training report summary
* Fix LossMetric batch size state
* Add NumericEntry de/serialize
* Fix clippy suggestion
* Compact recorder does not use compression (anymore)
* Add learner summary expected results tests
* Add summary to learner builder and automatically display in fit
- Add LearnerSummaryConfig
- Keep track of summary metrics names
- Add model field when displaying from learner.fit()
Dirley Jordan