Fibre is described as a nutrient, which is misleading. Humans cannot break most of it down, and its effects come from what happens to it after our own digestion has finished.
Human enzymes cannot cut these bonds
Starch and fibre are both chains of sugar units, and the difference lies in how those units are joined together.
Our digestive enzymes are shaped to cut one type of linkage and not the other, so starch is broken into absorbable sugars while fibre passes through unchanged.
Which means fibre arrives in the large intestine essentially as it was eaten, still carrying the energy locked in its bonds.
Gut bacteria have the enzymes we lack
The microbial population of the large intestine collectively carries a far wider enzyme range than the human genome encodes.
Those organisms ferment fibre, and in doing so release short-chain fatty acids that the cells lining the colon use directly as fuel.
So a portion of fibre's energy does reach us, indirectly, having passed through an organism that was able to open a door we cannot.
Soluble and insoluble do different jobs
Soluble fibre dissolves and forms a gel, which thickens gut contents, slows the movement of sugars into the bloodstream and gives bacteria something readily fermentable.
Insoluble fibre does not dissolve and mostly adds bulk and structure, which affects transit time through the gut more than it affects fermentation.
Most plant foods contain both in varying proportion, which is why the categories are more useful for understanding mechanisms than for choosing between foods.
Processing removes it before cooking does
Refining grains strips the outer layers where most fibre sits, which is the main difference between a whole grain and a refined one.
Juicing removes it entirely, and fine milling breaks it into smaller pieces that behave differently even though the quantity is unchanged.
Ordinary cooking, by contrast, softens fibre without removing it, so a cooked vegetable retains what a peeled or juiced one loses.
Increases are best made gradually
A sudden large increase gives the existing bacterial population more material than it is adapted to process, and the fermentation that follows produces gas and discomfort.
The population shifts over weeks in response to what it is fed, so a gradual increase gives that adjustment time to happen.
Anyone with a diagnosed digestive condition should treat this as a question for a clinician, since several conditions respond to fibre in ways that run against general advice.