Walls made of straw, clay and wood deteriorate when exposed to too much moisture over time. If you want your building to last, you should get your walls up off the ground and onto a water-resistant foundation.
Stone was the traditional material for foundations in many parts of the world. Stone foundations can either be carefully dry-stacked with no mortar, or else mortared for stability and to keep weather and animals from getting through the cracks. In a seismic zone, a reinforced concrete foundation is a prudent choice, but it can still be lined with stone to make it look more natural.
I have experimented with many other kinds of foundations for natural buildings. One my favorite materials for this purpose is “urbanite” or recycled concrete chunks, which can be stacked and assembled much like stone. Another is recycled feed sacks filled with gravel and tied together with barbed wire. This is a variation on the building system known as “earthbags” or “superadobe.” Other foundation materials I’ve worked with include recycled tires rammed with earth, and rot-resistant wood such as redwood. The most suitable material for your building depends on the specifics of your design and what materials are locally available.