Welcome to the Forum.. No doubt others will chip-in soon.. You could use anything fairly solid, that takes your fancy.. Log-roll, Roofing slates/tiles, thin-section natural stone (on edge), small paving (again, on edge), brick/stone walling, Large 'rockery-type' stones, some form of lawn-edging, decking timber (on edge).. - You get the idea?
Welcome to GSC! Lots of good ideas from Phil already, nothing much to add to that, at a slight tangent I'd suggest you ensure that the bottom of the "canyon" is well drained - you don't want your track sitting in a couple of inches of water every time it rains....
Any of Phils ideas will work, broken Paving Slabs will make a nice feature broken up very small and back filled with cement to the lawn should make them fairly permanent. Leave an inch below the Grass Line to be filled with soil and the grass will grow to the ends.. Or do a mowing border with Block Pvious cemented in for a nice neat lawn edge.
May I add, trimming the grass at the canyon's edge might be an issue.
As JonD mentioned, leave the retaining wall, if you build one, below the grass line. This will make cutting the grass at the very edge somewhat simpler, as you can run the front of the mower over the edge. If you raise the retaining wall above the edge of the cut, then a string trimmer is in order. Personally, I have a difficult time with string trimmers.
Also, not to put a damper on your plans, but as Gizzy mentioned, debris likes gullies.