* Nonzero should return an empty vec for zero tensors
* Add nonzero empty test
* Add missing import
---------
Co-authored-by: Nathaniel Simard <nathaniel.simard.42@gmail.com>
* implemented muli-dim index for GatherNode
The `NodeCodegen` impl for `GatherNode` now performs gather in complete
accordance with the ONNX Gather spec.
- a `gather` function was added to the gather.rs file
- `gather()` is now called within the codegen instead of `tensor.select()`
- a test with two test cases have been added
- test axes 0 and 1
- both use 2D index tensors
* add gather_onnx to numeric api
Added int and float implementations of gather to the burn-tensor numeric
api:
- named the methods `gather_onnx` to not be confused with the current
`gather`
- these implementations follow the `Gather` ONNX spec
Updated the gather*.py variants and their onnx outputs
* modified files didn't end up in last commit
* tests passing for onnx gather
The implementation of gather for the ONNX `Gather` spec is tentatively
complete:
- py test models are updated
- onnx_tests are modified and passing: `gather`, `gather_scalar`, and
`gather_shape`
- node/gather tests are passing
NOTE: The two additional tests in crates/burn-import/src/burn/node/gather.rs that test
the actual functionality of gather are likely to be deleted, since they
are redundant to the tests in
crates/burn-import/onnx-tests/tests/onnx_tests.rs.
* inlined onnx gather within codegen
* rm gather_onnx from public api; rm unnecessary tests
* add comments to gather py models
* some codegen changes; formatting to appease run-checks
- Some necessary changes and improvements to the codegen inlined code
after translating from public api (removed in previous commit).
- Changed some formatting that run-checks complained about.
* simplify gather codegen; include 1d and 2d onnx tests
Modified the `Gather` codegen per requested changes:
- combined match statements on index
- remove use of `alloc::vec::Vec`
- use map -> collect instead of procedural
- include a 1d index gather onnx test
- remove superflous tests
* delete unused gather.onnx
* Add Hard Sigmoid activation function
* Add ONNX import conversion for HardSigmoid
* Update supported operators list
* Update book
* Make test comparison approximate to eliminate precision issues
* Add burn-candle test
* Fix name in E2E test generator
* renaming repeat to repeat_dim
* implementing repeat function
* renaming repeat files to repeat_dim
* renaming part 2
* renaming part 3
* renaming part 4
* renaming part 5
* adding test file
* adding unit test
* adding rust book documentation
* adding function args doc
* fixing tests
* changing repeat api to match pytorch equivalent
* fixing clippy error
* Implement ONNX pad
* ONNX pad arguments fix
pad now requires 2 or more arguments
if the third argument is not given, it will default to 0
* fixing bug in input len fix
* change panic comment
Change panic comment from needing two inputs. This comes from the fact that the ONNX spec requires two necessary inputs but could have more two more optional argument.
---------
Co-authored-by: JC <you@example.com>
Co-authored-by: mepatrick73 <pameu17@ulaval.ca>
* Move QuantizationScheme to burn-tensor
* Refactor QuantizedTensorPrimitive to include the quantization strategy
* Fix QFloat tensor data display
* Refactor quantization methods to use scheme and qparams (on backend device)
* Fix clippy
* Fix fmt
* Add qtensor primitive tests
* Add QuantizationBackend, QTensorOps and QTensor
* Refactor QTensorOps as part of Backend trait
* Add tensor dequantize, QFloat dtype and default affine/symmetric quant
* Add ndarray default quantization implementation
* Fix clippy
* Add rayon parallel iter
* Add quantization operations to book
* Add q_shape and q_device ops to avoid converting the tensor just to get attributes
* Implement autodiff grad ops
* Mark autodiff todo for QAT
* Remove note
* Add q_inner and q_from_inner
* Implement 3D and transposed 3D convolutions.
* Merge changes from onnx-ir #1921 pr
---------
Co-authored-by: Dilshod Tadjibaev <939125+antimora@users.noreply.github.com>
* Move distribution to module
* Add new TensorData with serialization support
* Implement display and from for TensorData
* Add missing Cargo.lock
* Add missing bytemuck feature
* Add zeros, ones, full and random TensorData methods
* Refactor Data -> TensorData usage
* Fix tests
Since TensorData is not generic over the element type anymore no type inference can be done by the compiler. We must explicitly cast the expected results to the expected backend type.
* Remove commented line
* Fix import
* Add record-backward-compat
* Remove dim const generic from TensorData
* Support NestedValue de/serialization with TensorData
* Fix burn-jit tests
* Remove eprinln
* Refactor onnx import to use TensorData
* Fix tch from_data
* Fix nested value serialization for u8
* Fix missing import
* Fix reduce min onnx test
* Fix deprecated attribute
* Remove shape getter
* Remove strict assert in tests
* Add tensor data as_bytes
* Add tensor check for rank mismatch
* Fix typo (dimensions plural)
* Fix error message
* Update book examples with from_data and fix Display impl for TensorData
* Add deprecation note
* Element already implements One
* Add element module
* Add our own traits for Zero, One and ToPrimitive to support bool Element
* Fix typo
* Add basic tests for ToPrimitive with expected values
* The most important change of all
* Remove One + Zero identities
* Move zero/one outside mapv + refactor ToPrimitive -> ToElement trait
* Add num-traits to NOTICES.md
* Updated documentation for unfold4d
Added links between the struct and the config. Added a link to the related burn_tensor function in the documentation for the forward function.
* Changing nn relu module documentation to functional api
Removing the formula for relu from the module API to the functional API,
citing a paper relevant to relu
and mentionning the functional API in the module API
* Linking gelu module API documentation to functional API documentation
* Linear module : adding documentation
Adding documentation to the Linear module
mentionning that LinearConfig struct
should be used when creating a Linear Layer
Also adding links to the documentation that points people toward
the right path
* Updated documentation for dropout
Added links between the struct and the config. Added a link to the struct in the forward function for more info.
* embedding + swiglu
* RotaryEncodying : adding documentation
Adding documentation stating the RotaryEncoding should be created using a RotaryEncodingConfig
* prelu: adding documentation
Adding documentation to the prelu module:
- Linking forward function documentation to the functional API
- Citing the first paper to mention prelu
- Adding documentation saying that prelu layer should be created using PReluConfig
* pos_encoding: adding documentation
* Updated documentation for mha
Added links for more info. Added shape info at some places.
* docs: Add documentation for Gru module
Provide documentation for the Gru module, including its configuration and usage. Include a link to the paper that introduced the Gated Recurrent Unit (GRU) and specify that the module should be created using GruConfig. Also, mention that the forward function returns a state tensor with specific dimensions.
* burn-core-nn-transformers: adding documentation
Adding documentation:
- Says to use config to create the layers
- Add mathematical formula to the pwff forward pass
- Add citation in the pwff to the "Attention is all you need" paper
* Updated documentation: ConvTranspose1d and ConvTranspose2d
* docs: Add documentation for Lstm and BiLstm modules
Provide documentation for the Lstm and BiLstm modules, including their configurations and usage. Include links to the papers that introduced Long Short-Term Memory (LSTM) and Bidirectional LSTM. Specify that the modules should be created using LstmConfig and BiLstmConfig respectively.
* docs: Update documentation for ConvTranspose1d and ConvTranspose2d modules
* loss: Adding documenntation to the loss layers
Adding documentation stating to use the config to create the layer
* chore: Refactor Conv1d module imports and update documentation
* docs: Add documentation for AdaptiveAvgPool1d and AdaptiveAvgPool2d modules
Added references to the burn_tensor associated functions. Added links between the struct and the config.
* Refactor Conv1d module imports and update documentation
* chore: Refactor Conv2d module imports and update documentation
* Add documentation for AvgPool1d and AvgPool2d modules
Added references to the burn_tensor associated functions. Added links between the struct and the config.
* Add documentation for MaxPool1d and MaxPool2d modules
Added references to the burn_tensor associated functions. Added links between the struct and the config.
* Add documentation for leaky_relu and removed Config generic
Added references to the burn_tensor associated functions. Added links between the struct and the config. Removed the backend generic from the config since it's not needed (might be a breaking change).
* refactor: Update BatchNormConfig initialization and add documentation.
* Added link to config in embedding struct documentation
* refactor: Update GroupNormConfig initialization and add documentation
* refactor: Update InstanceNormConfig initialization and add documentation
* feat: Update LayerNormConfig initialization and add documentation
* refactor: Update RmsNormConfig initialization and add documentation
* fixed: removed #derive accidentally
* Added missing backticks in pools' shapes
* Format nn doc
* Make config fields public in nn modules
* Update import statements in nn modules
Changed burn_tensor imports to crate::tensor
* Update import statements in nn modules' tests
Changed burn_tensor imports to crate::tensor
* breaking change refactor: Update GroupNormConfig and InstanceNormConfig initialization
* Make SwiGlu fields public
* grammar
* slashes
* input tensors grouping
* copy-pasta mistake
* a not an >:I
* Capitalization
* better desc
* math 'n ticks
* group_norm functional implementation
* removed the ... struct
* decoder typo
* fmt
* referring to private fn in docs
---------
Co-authored-by: Thierry Cantin-Demers <piertcd@gmail.com>
Co-authored-by: mepatrick73 <pameu17@ulaval.ca>
* #1747
Upgrade Rust dependencies
* Revert upgrade for tch
The update of tch on windows gives an error:
INTEL MKL ERROR: The specified module could not be found. mkl_vml_avx2.1.dll.
Intel MKL FATAL ERROR: cannot load mkl_vml_avx2.1.dll or mkl_vml_def.1.dll.
* Keep only .cargo/config.toml file which works with rust > 1.75
---------
Co-authored-by: Sylvain Benner <sylvain@benner.online>
Nathaniel Simard