| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[torch-mlir][test] cleanup trailing whitespace in mlir files (#2806) | 2 年前 | |
Re-enable torch-adjust-calling-conventions tests (#4034) This PR updates AdjustCallingConventionsPass to the dialect conversion framework API updates introduced in https://github.com/llvm/llvm-project/pull/116470. This may not be an optimal use of the new API, but it is functional. Suggestions welcome! fixes #3983 | 1 年前 | |
[Torch] Canonicalize pool ops with single int tuple params. (#4250) Fixes https://github.com/llvm/torch-mlir/issues/3885 by repeating the single int to match with expected spatial dims. | 8 个月前 | |
handles 2,3,4 from https://github.com/llvm/torch-mlir/issues/1963 (#1964) | 3 年前 | |
[Torch] Fold aten.to.dtype on splat constants. (#4306) This commit teaches AtenToDtypeOp::fold to constant-fold dtype conversions when the operand is a splat DenseElementsAttr. Folding is done according to torch's rounding behavior, i.e. * Bool: 0 and -0.0 → false; nonzero/NaN/±Inf → true. * Float → Int: round toward zero. * Int → Float: sign-aware, rmNearestTiesToEven. * Float ↔ Float: use builtin mlir::FloatType::getFloatSemantics(). * Int ↔ Int: use zextOrTrunc / sextOrTrunc based on source signedness. Folding is only performed when non_blocking == false, copy == false, and memory_format is None. | 10 个月前 | |
[custom op] Generalize shape library logic to work with dtypes (#1594) * [custom op] Generalize shape library logic to work with dtypes This commit generalizes the shape library logic, so that dtype rules for ops can also be expressed using the same mechanism. In other words, each op can now have a shape function and a dtype function specified in Python that is imported during lowering to calculate the shapes and dtypes throught a program. For more information about how to specify a dtype function, see the updated docs/adding_a_shape_and_dtype_function.md. For those not familiar with how the shape library works, the file docs/calculations_lib.md provides an overview. | 3 年前 | |
Iteratively run the main simplification pipeline. This introduces a new pass LowerToBackendContract (better name very welcome) which performs the bulk of the simplifications that we do, such as - shape refinement - dtype refinement - maximizing value semantics - inlining global slots - decomposing complex ops The key difference from before is that it iterates the set of transformations, which can help to break a number of "catch-22" issues where one simplification depends on another, the latest example being here: https://github.com/llvm/torch-mlir/issues/1131 This also exposed that RefineTypes was sometimes crashing/asserting for certain inputs. This commit hardens it a bit. | 3 年前 | |
[TorchDialicet] Fix bug for AtenOnes/AtenZeros/AtenFullOp fold function (#4320) Co-authored by: xinyi.chen@terapines.com Fix bug for AtenOnes/AtenZeros/AtenFullOp fold function. Specify using mlir namespace for Integer/FloatType which was unexpected used in torch namespace. | 10 个月前 | |
Rework how global slot initializers work. Rather than a per-global-slot initializer region, we now have one for the whole module. For example, it might look like this: torch.global_slot "private" @tensor : !torch.tensor torch.global_slot "private" @list : !torch.list<tensor> torch.global_slot.module_initializer { %0 = torch.tensor.literal(dense<0.0> : tensor<f32>) : !torch.tensor %1 = torch.prim.ListConstruct %0 : (!torch.tensor) -> !torch.list<tensor> torch.initialize.global_slots [ @tensor(%0 : !torch.tensor) @list(%1 : !torch.list<tensor>) ] } This new structure allows GlobalizeObjectGraph to create the initializer in a much simpler way, avoiding the need to reason about whether different slots alias each other. Reasoning about whether slots alias each other now is the responsibility of InlineGlobalSlots, which has to do a much more complicated analysis, implemented using MLIR's dataflow analysis framework. Recommended review order: - Check out the new IR constructs in the .mlir files of various passes - Op definitions (*.td) - Changes to GlobalizeObjectGraph pass. - InlineGlobalSlots pass (~total rewrite) - Misc changes: - Moving torchMlirAdjustStaticInformation for sharing with C++ code. - EraseModuleInitializer pass To make this a bit nicer, it would be good to have a torch.module op with an initializer region attached. That would be more invasive though. This change has highlighted certain aspects of our project layering which are worth calling out. None of our backends can handle global slots, so we enforce that there are no global slots before backend lowering. At an earlier stage in the project, we had aspirations of transparently handling mutable global state and such, but for reasons described below, that is no longer a goal. So really global slots should be seen as a progressive lowering step as part of inlining all the IValue's in the original program (GlobalizeObjectGraph is also one such step). Over time, with insights from work like IREE-JAX, it has become clear that there isn't a reliable programming model we can compile for users where we just transparently handle mutable global state (and some other things, like lists and dictionaries). There is a need for an "outer program" that orchestrates more restricted subroutines of the kind we can handle in our compile flow here. The benefit of that is that it decouples considerations like shapes, dtypes, etc. from the program constructs used in the outer program. As long as the outer program can efficiently invoke (pipelining/async/etc.) high-performance data-parallel numerical subroutines of the kind we compile in our flow here, then there is a complete programming model. This is also consistent with the direction of upstream PyTorch which is becoming more tracing-based (which inherently loses a lot of program structure, which then has to be applied back with an "outer program" orchestrating the traced subroutines). | 3 年前 | |
Rework how global slot initializers work. Rather than a per-global-slot initializer region, we now have one for the whole module. For example, it might look like this: torch.global_slot "private" @tensor : !torch.tensor torch.global_slot "private" @list : !torch.list<tensor> torch.global_slot.module_initializer { %0 = torch.tensor.literal(dense<0.0> : tensor<f32>) : !torch.tensor %1 = torch.prim.ListConstruct %0 : (!torch.tensor) -> !torch.list<tensor> torch.initialize.global_slots [ @tensor(%0 : !torch.tensor) @list(%1 : !torch.list<tensor>) ] } This new structure allows GlobalizeObjectGraph to create the initializer in a much simpler way, avoiding the need to reason about whether different slots alias each other. Reasoning about whether slots alias each other now is the responsibility of InlineGlobalSlots, which has to do a much more complicated analysis, implemented using MLIR's dataflow analysis framework. Recommended review order: - Check out the new IR constructs in the .mlir files of various passes - Op definitions (*.td) - Changes to GlobalizeObjectGraph pass. - InlineGlobalSlots pass (~total rewrite) - Misc changes: - Moving torchMlirAdjustStaticInformation for sharing with C++ code. - EraseModuleInitializer pass To make this a bit nicer, it would be good to have a torch.module op with an initializer region attached. That would be more invasive though. This change has highlighted certain aspects of our project layering which are worth calling out. None of our backends can handle global slots, so we enforce that there are no global slots before backend lowering. At an earlier stage in the project, we had aspirations of transparently handling mutable global state and such, but for reasons described below, that is no longer a goal. So really global slots should be seen as a progressive lowering step as part of inlining all the IValue's in the original program (GlobalizeObjectGraph is also one such step). Over time, with insights from work like IREE-JAX, it has become clear that there isn't a reliable programming model we can compile for users where we just transparently handle mutable global state (and some other things, like lists and dictionaries). There is a need for an "outer program" that orchestrates more restricted subroutines of the kind we can handle in our compile flow here. The benefit of that is that it decouples considerations like shapes, dtypes, etc. from the program constructs used in the outer program. As long as the outer program can efficiently invoke (pipelining/async/etc.) high-performance data-parallel numerical subroutines of the kind we compile in our flow here, then there is a complete programming model. This is also consistent with the direction of upstream PyTorch which is becoming more tracing-based (which inherently loses a lot of program structure, which then has to be applied back with an "outer program" orchestrating the traced subroutines). | 3 年前 | |
Integrate LLVM at 6d38dbf6eb56fd2b3399565af455de96a99ffa0f (#4103) Update LLVM to https://github.com/llvm/llvm-project/commit/72144d119a7291f8b6b8e022a2947fbe31e66afc TOSA Updates Summary: 1: [TOSA] Update rescale input_/output_zp and double_round attribute Update tosa.rescale input_/output_zp as inputs according to TOSA 1.0 Update double_round bool attribute to rounding_mode in alignment with TOSA 1.0. rounding_mode supports "SINGLE_ROUND", "INEXACT_ROUND", and "DOUBLE_ROUND". Existing double_round behaviours are mapped as followed: double_round = true -> rounding_mode = "DOUBLE_ROUND" double_round = false -> rounding_mode = "SINGLE_ROUND" 2: [TOSA] Update tosa.negate's zero-points to inputs Update LIT tests and XFAIL sets 3: [TOSA] Update tosa.int_div to tosa.intdiv Update LIT tests Signed-off-by: Vivek Khandelwal <vivekkhandelwal1424@gmail.com> Co-authored-by: Justin Ngo <justin.ngo@arm.com> | 1 年前 | |
lib/Dialect/Torch/IR/TorchOps.cpp: fix: use-after-free: erasing an operation during folding (#4274) This fixes a SEGFAULT in the GreedyPatternRewriteDriver and adds a missing size check to the torch.aten._assert_tensor_metadata operation. Erasing an operation during folding is not allowed. Folding the operation may eithermodify it in place or return a set of replacements, but may not erase the operation. (see https://github.com/llvm/llvm-project/blob/e56384ff540e68f9d0500fa27a95354c0730e37b/mlir/lib/Transforms/Utils/GreedyPatternRewriteDriver.cpp#L492-L508) Doing this causes a SEGFAULT (witnessed on macOS Sequoia 15.5, Apple M4): Stack dump: 0. Program arguments: build/bin/torch-mlir-opt -canonicalize --split-input-file -verify-diagnostics test/Dialect/Torch/invalid_canonicalize.mlir #0 0x0000000104091524 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (build/bin/torch-mlir-opt+0x10140d524) #1 0x000000010408fa5c llvm::sys::RunSignalHandlers() (build/bin/torch-mlir-opt+0x10140ba5c) #2 0x0000000104091bc8 SignalHandler(int, __siginfo*, void*) (build/bin/torch-mlir-opt+0x10140dbc8) #3 0x0000000181e10624 (/usr/lib/system/libsystem_platform.dylib+0x1804ac624) #4 0x0000000103c1f7a8 (anonymous namespace)::GreedyPatternRewriteDriver::processWorklist() (build/bin/torch-mlir-opt+0x100f9b7a8) #5 0x0000000103c1cf4c mlir::applyPatternsGreedily(mlir::Region&, mlir::FrozenRewritePatternSet const&, mlir::GreedyRewriteConfig, bool*) (build/bin/torch-mlir-opt+0x100f98f4c) #6 0x0000000102c8f62c (anonymous namespace)::Canonicalizer::runOnOperation() (build/bin/torch-mlir-opt+0x10000b62c) #7 0x0000000103c72fa4 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) (build/bin/torch-mlir-opt+0x100feefa4) #8 0x0000000103c750d4 mlir::PassManager::run(mlir::Operation*) (build/bin/torch-mlir-opt+0x100ff10d4) #9 0x0000000102c8d774 performActions(llvm::raw_ostream&, std::__1::shared_ptr<llvm::SourceMgr> const&, mlir::MLIRContext*, mlir::MlirOptMainConfig const&) (build/bin/torch-mlir-opt+0x100009774) #10 0x0000000102c8d35c llvm::LogicalResult llvm::function_ref<llvm::LogicalResult (std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>::callback_fn<mlir::MlirOptMain(llvm::raw_ostream&, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, mlir::DialectRegistry&, mlir::MlirOptMainConfig const&)::$_0>(long, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&) (build/bin/torch-mlir-opt+0x10000935c) #11 0x000000010403194c mlir::splitAndProcessBuffer(std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::function_ref<llvm::LogicalResult (std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>, llvm::raw_ostream&, llvm::StringRef, llvm::StringRef)::$_0::operator()(llvm::StringRef) const (build/bin/torch-mlir-opt+0x1013ad94c) #12 0x00000001040316a4 mlir::splitAndProcessBuffer(std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::function_ref<llvm::LogicalResult (std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>, llvm::raw_ostream&, llvm::StringRef, llvm::StringRef) (build/bin/torch-mlir-opt+0x1013ad6a4) #13 0x0000000102c87078 mlir::MlirOptMain(llvm::raw_ostream&, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>, mlir::DialectRegistry&, mlir::MlirOptMainConfig const&) (build/bin/torch-mlir-opt+0x100003078) #14 0x0000000102c8731c mlir::MlirOptMain(int, char**, llvm::StringRef, llvm::StringRef, mlir::DialectRegistry&) (build/bin/torch-mlir-opt+0x10000331c) #15 0x0000000102c87538 mlir::MlirOptMain(int, char**, llvm::StringRef, mlir::DialectRegistry&) (build/bin/torch-mlir-opt+0x100003538) #16 0x0000000102c85cd0 main (build/bin/torch-mlir-opt+0x100001cd0) #17 0x0000000181a36b98 build/tools/torch-mlir/test/Dialect/Torch/Output/invalid_canonicalize.mlir.script: line 1: 72586 Segmentation fault: 11 build/bin/torch-mlir-opt -canonicalize --split-input-file -verify-diagnostics test/Dialect/Torch/invalid_canonicalize.mlir Since the torch.aten._assert_tensor_metadata operation is only used for static assertion during compile time the folding can be replaced by a canonicalization that checks the assert and then uses a rewriter to erase the operation. The second commit deals with a missing size check in the assert operation before using a zip operation. Without the explicit checkout of the size, the would assert not fail in case the size of the dimensions were the same, but there are either less or more dimensions in the input than specified in the assert. --------- Signed-off-by: Florian Walbroel <walbroel@roofline.ai> Co-authored-by: Florian Walbroel <walbroel@roofline.ai> | 1 年前 | |
Allow running DecomposeComplexOps more than once (#1671) The current implementation of DecomposeComplexOps fails if an op expected to be decomposed does not get decomposed in the first iteration of the createTorchSimplificationPipeline in LowerToBackendContractPass. However, some graphs require multiple iterations of createTorchSimplificationPipeline to fully propagate all statically knowable information, such as dtypes and shapes, to the entire graph, sometimes resulting in the need to run DecomposeComplexOps more than once. This commit changes DecomposeComplexOps to use a greedy algorithm for pattern application and moves the legalization check of ops to the LowerToBackendContractPass to allow for the DecomposeComplexOps to run more than once. | 3 年前 | |
[torch] Basic support for per-channel quantized graphs (#3623) This patch adds basic support for lowering graphs with per-channel quantization. Per-channel quantized ops have to be excluded from FuseQuantizedOps for now but can be used in QDQ quantized form. Using this patch, we're able to import and execute (on the linalg backend) graphs with per-channel quantization applied using the "new" PyTorch 2.0 Export Quantization. | 1 年前 | |
Add alias analysis for cast-like ops to maximize-value-semantics (#2160) When use_tracing=True is used to import a model into Torch-MLIR, several casts get inserted in the IR to bridge the untyped inputs and outputs with the typed body of the computation. These casts create extra aliases of tensors that cause the current analysis in maximize-value-semantics to fail. In particular, the maximize-value-semantics analysis assumes that the only valid alias right after an overwrite is the overwritten alias. So, if there is a use of a casted version of the overwritten alias after the overwrite, the analysis fails. This commit improves the analysis by identifying all cast-like aliases of the overwritten alias and allowing such aliases to be used after an overwrite. Because this issue only arises when using tracing, it cannot be currently tested e2e, so only lit test is added. | 3 年前 | |
Register fake_quantize related ops (#3522) Register aten.fake_quantize_per_channel_affine and aten.fake_quantize_per_tensor_affine.tensor_qparams ops --------- Co-authored-by: Ze Zhang <ze.zhang@getcruise.com> | 2 年前 | |
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that func and call operations refer to the func dialect 2. avoid duplicate registration of dialects | 4 年前 | |
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that func and call operations refer to the func dialect 2. avoid duplicate registration of dialects | 4 年前 | |
build: manually update PyTorch version (#3627) Set PyTorch and TorchVision version to nightly release 2024-08-18. This commit also updates the scaled_dot_product_attention op. A new attribute enable_gqa has been added. As of now, only the default value for the same is supported. Signed-Off By: Vivek Khandelwal <vivekkhandelwal1424@gmail.com> | 1 年前 | |
Support DerefineOp in RefinePublicReturn. | 3 年前 | |
[ONNX][TORCH] Add Onnx->Linalg lowering for RotaryEmbedding Op (#4002) This commit adds the Onnx->Linalg lowering for Onnx's RotaryEmbedding op (ref: https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#commicrosoftrotaryembedding) by registering a customized torch op named OnnxVariantAtenRotaryEmbeddingOp. This is done so that the Onnx's RotaryEmbedding op can be lowered to this op and this op can be lowered from Torch->Linalg. The lowering has been adopted from the OnnxRuntime. Files for references: 1.) https://github.com/microsoft/onnxruntime/blob/e1e3f623f61816008e79dddc91a51ffe7f0ff5cf/onnxruntime/contrib_ops/cpu/bert/rotary_embedding.cc#L47-L93 2.) https://github.com/microsoft/onnxruntime/blob/94c69f55d480cb4a8dcbc161d29ef3acca9392a7/onnxruntime/contrib_ops/cpu/bert/rotary_embedding_helper.h --------- Signed-off-by: Vivek Khandelwal <vivekkhandelwal1424@gmail.com> Co-authored-by: zjgarvey <47986913+zjgarvey@users.noreply.github.com> | 1 年前 | |
[ONNX][TORCH] Add Onnx->Linalg lowering for RotaryEmbedding Op (#4002) This commit adds the Onnx->Linalg lowering for Onnx's RotaryEmbedding op (ref: https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#commicrosoftrotaryembedding) by registering a customized torch op named OnnxVariantAtenRotaryEmbeddingOp. This is done so that the Onnx's RotaryEmbedding op can be lowered to this op and this op can be lowered from Torch->Linalg. The lowering has been adopted from the OnnxRuntime. Files for references: 1.) https://github.com/microsoft/onnxruntime/blob/e1e3f623f61816008e79dddc91a51ffe7f0ff5cf/onnxruntime/contrib_ops/cpu/bert/rotary_embedding.cc#L47-L93 2.) https://github.com/microsoft/onnxruntime/blob/94c69f55d480cb4a8dcbc161d29ef3acca9392a7/onnxruntime/contrib_ops/cpu/bert/rotary_embedding_helper.h --------- Signed-off-by: Vivek Khandelwal <vivekkhandelwal1424@gmail.com> Co-authored-by: zjgarvey <47986913+zjgarvey@users.noreply.github.com> | 1 年前 | |
convert to double before float materialization in scalarize shapes (#3887) Addresses a bug when trying to materialize a non fp64 attr to a constant float op in scalarize shapes. | 1 年前 | |
[MLIR][TORCH] Add E2E support for view_as_real op (#2419) * view_as_real test case, allow dtype in testutils.randn * abstract python upstream func implemented * fixed upstream dtype func, implemented view_as_real backend op * formatted AtenViewAsRealOp, removed change in e2etest/framework * removed test suit from reshape_like.py, because it's moved to basic.py * implemented C-API wrapper for mlirComplexF128 type * fixed torch.complex dtype width in MLIR and Torch MLIR, deleted float16 dtype dict * Changed IR input of aten fft_fft unit test * code refactored * code refactored and fixed ci test * refactored: removed white spaces, and rolled back to having both input/output affine expr * refactored: deleted output affine expr to reduce redundancy * xfail ltc backend * removed ComplexImag and ComplexReal from torchdynamo xfail set * copied and pasted from main branch as there's no change to be made in this file * refactored abstract_interp_lib_gen.py * refactored: torchtypes.td, formatted, removed commented out code | 2 年前 | |
[MLIR][TORCH] Only unroll prim loop-like ops within a torch.shape.calculate region (#3812) Reports a match failure for the pattern FullyUnrollPrimLoop when the loop op is not in a region defined by a torch.shape.calculate op. This is needed to avoid unrolling prim loops generated by ONNX IR, since we are applying shape refinement in the torch-onnx-to-torch-backend-pipeline introduced in fa4794d . See also the discussion in <https://github.com/iree-org/iree/pull/18867#discussion_r1811101655> | 1 年前 | |
[TORCH] Add f8 support in getConstantWithGivenDtypeAndValue utility (#4148) Fixes https://github.com/iree-org/iree/issues/20570. --------- Signed-off-by: Vivek Khandelwal <vivekkhandelwal1424@gmail.com> | 1 年前 | |
[Torch] add fold logic for some ops (#3794) | 1 年前 | |
Add More Scalarize Shapes Patterns (#3810) ### new patterns: 1. Propagates aten.broadcast_to ops of a single value to an aten.full op 2. Propagates arithmetic operations through a templated class which associates some tensor arithmetic ops to their integer-scalar counterparts. These are a major blocker right now, since some models have a bunch of rank 0 arithmetic being done with tensor ops. See the lit test for an interesting example that pads an input to the smallest shape which will become divisible by twelve in dim0. If you think this is convoluted, you haven't been staring at ONNX generated IR long enough. 3. Adds a stronger folder for aten.eq.int to fold size.int == 0 to false. See the comment in that conversion pattern for more justification as to why it is acceptable to make this assumption here. This is another major blocker for models, since this lack of folding propagates to lack of folding for subsequent where.self operations. 4. Add AtenSqueezeDim to the existing FoldAtenSqueezeOpPattern ### other changes: 1. Add two new anchor ops: AtenArangeStartStepOp and Torch::RuntimeAssertOp. I've checked all possible sources of the runtime assert ops and it is always shape related. The Arange op only takes int inputs, and these are all shape related. Adds a size check to getting a list from literal ops. 2. Improved folders for int arithmetic ops to fold some common patterns. 3. adds the ability to get some values from scalar-tensor ops to getListFromTensor. 4. further cleans up getListFromTensor for readability. ### points to scrutinize: 1. I made the choice to scalarize div.Tensor (int dtype result) to floordiv.int. This is because our shape computations involving this kind of arithmetic are never negative in practice, and we don't have a "round towards zero" scalar int divide counterpart. 2. Anchoring on RuntimeAssertOp sounds really suspicious, and if someone happens to add a runtime assert in the future that doesn't boil down to shapes, then it would add to the worklist considerably. We might be able to get around this by adding "NoMemoryEffect" to ops which are "ReadOnly" so that the inputs for the runtime asserts get cse'd with existing elements of the worklist before we even get to this pass. | 1 年前 | |
Clean up verification of calling conventions. The implementation at this place was a remnent of the times the pipeline was run only once. Rely instead on the backend verification, after optimizations have had an opportunity to resolve some uncertainties. (e.g. !torch.optional). | 3 年前 | |
LowerToBackendContract: Explicitly error out on unimplemented operator (#1947) * LowerToBackendContract: Explicitly error out on unimplemented operator But only reject torch.operator when results are invalid. Otherwise it might be a custom op that the backend supports. | 3 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 2 年前 | ||
| 1 年前 | ||
| 8 个月前 | ||
| 3 年前 | ||
| 10 个月前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 10 个月前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 4 年前 | ||
| 4 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 3 年前 |