Value type abstraction for locations differ from others in that a Location can NOT be null. NOTE: dyn_cast returns an Optional<T>.
PiperOrigin-RevId: 220682078
Add static pass registration and change mlir-opt to use it. Future work is needed to refactor the registration for PassManager usage.
Change build targets to alwayslink to enforce registration.
PiperOrigin-RevId: 220390178
- simple perfectly nested band tiling with fixed tile sizes.
- only the hyper-rectangular case is handled, with other limitations of
getIndexSet applying (constant loop bounds, etc.); once
the latter utility is extended, tiled code generation should become more
general.
- Add FlatAffineConstraints::isHyperRectangular()
PiperOrigin-RevId: 220324933
Start of TFLite legalizer pass. Currently focussed on macro expanding ops, limited to what is registered directly in a separate pass (this should instead be a general pass), no querying of what gets produced, the matching is string based instead of using the ops proper (the matching TF ops should be defined) etc. This is a step to enable prototyping. In addition to the above shortcomings, the legalizer is very verbose in this form and should instead be driven by autogenerated patterns (same is true for the op builders too). But this starts from the explicit form and extracting out commonality in follow up.
Add definition for tfl.relu for basic selection of fused relu add.
PiperOrigin-RevId: 220287087
Adds equality constraints to dependence constraint system for accesses using dims/symbols where the defining operation of the dim/symbol is a constant.
PiperOrigin-RevId: 219814740
- Builds access functions and iterations domains for each access.
- Builds dependence polyhedron constraint system which has equality constraints for equated access functions and inequality constraints for iteration domain loop bounds.
- Runs elimination on the dependence polyhedron to test if no dependence exists between the accesses.
- Adds a trivial LoopFusion transformation pass with a simple test policy to test dependence between accesses to the same memref in adjacent loops.
- The LoopFusion pass will be extended in subsequent CLs.
PiperOrigin-RevId: 219630898
Introduce analysis to check memref accesses (in MLFunctions) for out of bound
ones. It works as follows:
$ mlir-opt -memref-bound-check test/Transforms/memref-bound-check.mlir
/tmp/single.mlir:10:12: error: 'load' op memref out of upper bound access along dimension tensorflow/mlir#1
%x = load %A[%idxtensorflow/mlir#0, %idxtensorflow/mlir#1] : memref<9 x 9 x i32>
^
/tmp/single.mlir:10:12: error: 'load' op memref out of lower bound access along dimension tensorflow/mlir#1
%x = load %A[%idxtensorflow/mlir#0, %idxtensorflow/mlir#1] : memref<9 x 9 x i32>
^
/tmp/single.mlir:10:12: error: 'load' op memref out of upper bound access along dimension tensorflow/mlir#2
%x = load %A[%idxtensorflow/mlir#0, %idxtensorflow/mlir#1] : memref<9 x 9 x i32>
^
/tmp/single.mlir:10:12: error: 'load' op memref out of lower bound access along dimension tensorflow/mlir#2
%x = load %A[%idxtensorflow/mlir#0, %idxtensorflow/mlir#1] : memref<9 x 9 x i32>
^
/tmp/single.mlir:12:12: error: 'load' op memref out of upper bound access along dimension tensorflow/mlir#1
%y = load %B[%idy] : memref<128 x i32>
^
/tmp/single.mlir:12:12: error: 'load' op memref out of lower bound access along dimension tensorflow/mlir#1
%y = load %B[%idy] : memref<128 x i32>
^
#map0 = (d0, d1) -> (d0, d1)
#map1 = (d0, d1) -> (d0 * 128 - d1)
mlfunc @test() {
%0 = alloc() : memref<9x9xi32>
%1 = alloc() : memref<128xi32>
for %i0 = -1 to 9 {
for %i1 = -1 to 9 {
%2 = affine_apply #map0(%i0, %i1)
%3 = load %0[%2tensorflow/mlir#0, %2tensorflow/mlir#1] : memref<9x9xi32>
%4 = affine_apply #map1(%i0, %i1)
%5 = load %1[%4] : memref<128xi32>
}
}
return
}
- Improves productivity while manually / semi-automatically developing MLIR for
testing / prototyping; also provides an indirect way to catch errors in
transformations.
- This pass is an easy way to test the underlying affine analysis
machinery including low level routines.
Some code (in getMemoryRegion()) borrowed from @andydavis cl/218263256.
While on this:
- create mlir/Analysis/Passes.h; move Pass.h up from mlir/Transforms/ to mlir/
- fix a bug in AffineAnalysis.cpp::toAffineExpr
TODO: extend to non-constant loop bounds (straightforward). Will transparently
work for all accesses once floordiv, mod, ceildiv are supported in the
AffineMap -> FlatAffineConstraints conversion.
PiperOrigin-RevId: 219397961
- Introduce Fourier-Motzkin variable elimination to eliminate a dimension from
a system of linear equalities/inequalities. Update isEmpty to use this.
Since FM is only exact on rational/real spaces, an emptiness check based on
this is guaranteed to be exact whenever it says the underlying set is empty;
if it says, it's not empty, there may still be no integer points in it.
Also, supports a version that computes "dark shadows".
- Test this by checking for "always false" conditionals in if statements.
- Unique IntegerSet's that are small (few constraints, few variables). This
basically means the canonical empty set and other small sets that are
likely commonly used get uniqued; allows checking for the canonical empty set
by pointer. IntegerSet::kUniquingThreshold gives the threshold constraint size
for uniqui'ing.
- rename simplify-affine-expr -> simplify-affine-structures
Other cleanup
- IntegerSet::numConstraints, AffineMap::numResults are no longer needed;
remove them.
- add copy assignment operators for AffineMap, IntegerSet.
- rename Invalid() -> Null() on AffineExpr, AffineMap, IntegerSet
- Misc cleanup for FlatAffineConstraints API
PiperOrigin-RevId: 218690456
This CL implements a very simple loop vectorization **test** and the basic
infrastructure to support it.
The test simply consists in:
1. matching the loops in the MLFunction and all the Load/Store operations
nested under the loop;
2. testing whether all the Load/Store are contiguous along the innermost
memory dimension along that particular loop. If any reference is
non-contiguous (i.e. the ForStmt SSAValue appears in the expression), then
the loop is not-vectorizable.
The simple test above can gradually be extended with more interesting
behaviors to account for the fact that a layout permutation may exist that
enables contiguity etc. All these will come in due time but it is worthwhile
noting that the test already supports detection of outer-vetorizable loops.
In implementing this test, I also added a recursive MLFunctionMatcher and some
sugar that can capture patterns
such as `auto gemmLike = Doall(Doall(Red(LoadStore())))` and allows iterating
on the matched IR structures. For now it just uses in order traversal but
post-order DFS will be useful in the future once IR rewrites start occuring.
One may note that the memory management design decision follows a different
pattern from MLIR. After evaluating different designs and how they quickly
increase cognitive overhead, I decided to opt for the simplest solution in my
view: a class-wide (threadsafe) RAII context.
This way, a pass that needs MLFunctionMatcher can just have its own locally
scoped BumpPtrAllocator and everything is cleaned up when the pass is destroyed.
If passes are expected to have a longer lifetime, then the contexts can easily
be scoped inside the runOnMLFunction call and storage lifetime reduced.
Lastly, whatever the scope of threading (module, function, pass), this is
expected to also be future-proof wrt concurrency (but this is a detail atm).
PiperOrigin-RevId: 217622889
out canonicalization pass to drive it, and a simple (x-x) === 0 pattern match
as a test case.
There is a tremendous number of improvements that need to land, and the
matcher/rewriter and patterns will be split out of this file, but this is a
starting point.
PiperOrigin-RevId: 216788604
Add target independent standard DMA ops: dma.start, dma.wait. Update pipeline
data transfer to use these to detect DMA ops.
While on this
- return failure from mlir-opt::performActions if a pass generates invalid output
- improve error message for verify 'n' operand traits
PiperOrigin-RevId: 216429885
mode. We even diagnose mistakes nicely (aside from the a/an vowel confusion
which isn't worth worrying about):
test/IR/invalid.mlir split at line tensorflow/mlir#399:8:34: error: 'note' diagnostic emitted when expecting a 'error'
%x = "bar"() : () -> i32 // expected-error {{operand defined here}}
^
PiperOrigin-RevId: 214773208
Super thin slice that can convert a MLIR program (with addfs) to MLIR HLO dialect. Add this as translations to mlir-translate. Also add hlo::AddOp op and HLO op registration.
PiperOrigin-RevId: 214480409
Instead of linking in different initializeMLIRContext functions, add a registry mechanism and function to initialize all registered ops in a given MLIRContext. Initialize all registered ops along with the StandardOps when constructing a MLIRContext.
PiperOrigin-RevId: 214073842
optimization pass:
- Give the ability for operations to implement a constantFold hook (a simple
one for single-result ops as well as general support for multi-result ops).
- Implement folding support for constant and addf.
- Implement support in AbstractOperation and Operation to make this usable by
clients.
- Implement a very simple constant folding pass that does top down folding on
CFG and ML functions, with a testcase that exercises all the above stuff.
Random cleanups:
- Improve the build APIs for ConstantOp.
- Stop passing "-o -" to mlir-opt in the testsuite, since that is the default.
PiperOrigin-RevId: 213749809
- Compress the identifier/kind of a Function into a single word.
- Eliminate otherFailure from verifier now that we always have a location
- Eliminate the error string from the verifier now that we always have
locations.
- Simplify the parser's handling of fn forward references, using the location
tracked by the function.
PiperOrigin-RevId: 211985101
terminators. Improve mlir-opt to print better location info in the split-files
case.
Before:
error: unexpected error: branch has 2 operands, but target block has 1
br bb1(%0tensorflow/mlir#1, %0tensorflow/mlir#0 : i17, i1)
^
after:
invalid.mlir split at line tensorflow/mlir#305:6:3: error: unexpected error: branch has 2 operands, but target block has 1
br bb1(%0tensorflow/mlir#1, %0tensorflow/mlir#0 : i17, i1)
^
It still isn't optimal (it would be better to have just the original file and
line number but is a step forward, and doing the optimal thing would be a lot
more complicated.
PiperOrigin-RevId: 211917067
inserting shape_casts as necessary.
Along the way:
- Add some missing accessors to the AtLeastNOperands trait.
- Implement shape_cast / ShapeCastOp standard op.
- Improve handling of errors in mlir-opt, making it easier to understand
errors when invalid IR is rejected by the verifier.
PiperOrigin-RevId: 211897877
- Make the tf-lower-control flow handle error cases better. Add a testcase
that (currently) fails due to type mismatches.
- Factor more code in the verifier for basic block argument checking, and
check more invariants.
- Fix a crasher in the asmprinter on null instructions (which only occurs on
invalid code).
- Fix a bug handling conditional branches with no block operands, it would
access &operands[0] instead of using operands.data().
- Enhance the mlir-opt driver to use the verifier() in a non-crashing mode,
allowing issues to be reported as diagnostics.
PiperOrigin-RevId: 211818291
Enable using GraphWriter to dump graphviz in debug mode (kept to debug builds completely as this is only for debugging). Add option to mlir-opt to print CFGFunction after every transform in debug mode.
PiperOrigin-RevId: 211578699
- Add a new -verify mode to the mlir-opt tool that allows writing test cases
for optimization and other passes that produce diagnostics.
- Refactor existing the -check-parser-errors flag to mlir-opt into a new
-split-input-file option which is orthogonal to -verify.
- Eliminate the special error hook the parser maintained and use the standard
MLIRContext's one instead.
- Enhance the default MLIRContext error reporter to print file/line/col of
errors when it is available.
- Add new createChecked() methods to the builder that create ops and invoke
the verify hook on them, use this to detected unhandled code in the
RaiseControlFlow pass.
- Teach mlir-opt about expected-error @+, it previously only worked with @-
PiperOrigin-RevId: 211305770
Outside of IR/
- simplify a MutableAffineMap by flattening the affine expressions
- add a simplify affine expression pass that uses this analysis
- update the FlatAffineConstraints API (to be used in the next CL)
In IR:
- add isMultipleOf and getKnownGCD for AffineExpr, and make the in-IR
simplication of simplifyMod simpler and more powerful.
- rename the AffineExpr visitor methods to distinguish b/w visiting and
walking, and to simplify API names based on context.
The next CL will use some of these for the loop unrolling/unroll-jam to make
the detection for the need of cleanup loop powerful/non-trivial.
A future CL will finally move this simplification to FlatAffineConstraints to
make it more powerful. For eg., currently, even if a mod expr appearing in a
part of the expression tree can't be simplified, the whole thing won't be
simplified.
PiperOrigin-RevId: 211012256
- for test purposes, the unroll-jam pass unroll jams the first outermost loop.
While on this:
- fix StmtVisitor to allow overriding of function to iterate walk over children
of a stmt.
PiperOrigin-RevId: 210644813
- Implement support for the TensorFlow 'If' op, the first TF op definition.
- Fill in some missing basic infra, including the ability to split a basic block, the ability to create a branch with operands, etc.
- Implement basic lowering for some simple forms of If, where the condition is a zero-D bool tensor and when all the types line up. Future patches will generalize this.
There is still much to be done here. I'd like to get some example graphs coming from the converter to play with to direct this work.
PiperOrigin-RevId: 210198760
- Have the parser rewrite forward references to their resolved values at the
end of parsing.
- Implement verifier support for detecting malformed function attrs.
- Add efficient query for (in general, recursive) attributes to tell if they
contain a function.
As part of this, improve other general infrastructure:
- Implement support for verifying OperationStmt's in ml functions, refactoring
and generalizing support for operations in the verifier.
- Refactor location handling code in mlir-opt to have the non-error expecting
form of mlir-opt invocations to report error locations precisely.
- Fix parser to detect verifier failures and report them through errorReporter
instead of printing the error and crashing.
This regresses the location info for verifier errors in the parser that were
previously ascribed to the function. This will get resolved in future patches
by adding support for function attributes, which we can use to manage location
information.
PiperOrigin-RevId: 209600980
Previously mlir-opt had initializeMLIRContext function that added certain ops to the OperationSet of the context. But for different tests we'd want to register different ops. Make initializeMLIRContext an extern function so that the context initialization/set of ops to register can be determined at link time. This allows out-of-tree operations to easily expand the custom parsing/printing while still using mlir-opt.
PiperOrigin-RevId: 209078315
- fix/complete forStmt cloning for unrolling to work for outer loops
- create IV const's only when needed
- test outer loop unrolling by creating a short trip count unroll pass for
loops with trip counts <= <parameter>
- add unrolling test cases for multiple op results, outer loop unrolling
- fix/clean up StmtWalker class while on this
- switch unroll loop iterator values from i32 to affineint
PiperOrigin-RevId: 207645967
- Implement a diagnostic hook in one of the paths in mlir-opt which
captures and reports the diagnostics nicely.
- Have the parser capture simple location information from the parser
indicating where each op came from in the source .mlir file.
- Add a verifyDominance() method to MLFuncVerifier to demo this, resolving b/112086163
- Add some PrettyStackTrace handlers to make crashes in the testsuite easier
to track down.
PiperOrigin-RevId: 207488548
- Sketch out a TensorFlow/IR directory that will hold op definitions and common TF support logic. We will eventually have TensorFlow/TF2HLO, TensorFlow/Grappler, TensorFlow/TFLite, etc.
- Add sketches of a Switch/Merge op definition, including some missing stuff like the TwoResults trait. Add a skeleton of a pass to raise this form.
- Beef up the Pass/FunctionPass definitions slightly, moving the common code out of LoopUnroll.cpp into a new IR/Pass.cpp file.
- Switch ConvertToCFG.cpp to be a ModulePass.
- Allow _ to start bare identifiers, since this is important for TF attributes.
PiperOrigin-RevId: 206502517
- Implement a full loop unroll for innermost loops.
- Use it to implement a pass that unroll all the innermost loops of all
mlfunction's in a module. ForStmt's parsed currently have constant trip
counts (and constant loop bounds).
- Implement StmtVisitor based (Visitor pattern)
Loop IVs aren't currently parsed and represented as SSA values. Replacing uses
of loop IVs in unrolled bodies is thus a TODO. Class comments are sparse at some places - will add them after one round of comments.
A cmd-line flag triggers this for now.
Original:
mlfunc @loops() {
for x = 1 to 100 step 2 {
for x = 1 to 4 {
"Const"(){value: 1} : () -> ()
}
}
return
}
After unrolling:
mlfunc @loops() {
for x = 1 to 100 step 2 {
"Const"(){value: 1} : () -> ()
"Const"(){value: 1} : () -> ()
"Const"(){value: 1} : () -> ()
"Const"(){value: 1} : () -> ()
}
return
}
PiperOrigin-RevId: 205933235
This patch adds support for basic block arguments including parsing and printing.
In doing so noticed that `ssa-id-and-type` is undefined in the MLIR spec; suggested an implementation in the spec doc.
PiperOrigin-RevId: 205593369
Add a default error reporter for the parser that uses the SourceManager to print the error. Also and OptResult enum (mirroring ParseResult) to make the behavior self-documenting.
PiperOrigin-RevId: 203173647
For checking parse errors, the input file is split and failures reported per memory buffer. Simply reporting the errors loses the mapping back to the original file. Change the reporting to instead relate the error reported back to the original file.
Use SourceMgr's PrintMessage consistently for errors and relates back to file being parsed.
PiperOrigin-RevId: 202136152
Add diagnostic reporter function to lexer/parser and use that from mlir-opt to report errors instead of having the lexer/parser print the errors.
PiperOrigin-RevId: 201892004
Add parsing tests with errors. Follows direct path of splitting file into test groups (using a marker) and parsing each section individually. The expected errors are checked using FileCheck and parser error does not result in terminating parsing the rest of the file if check-parser-error.
This is an interim approach until refactoring lexer/parser.
PiperOrigin-RevId: 201867941
