Abstract: We consider the problem of testing statistical hypotheses and building confidence sequences for elicitable and identifiable functionals, a broad class of statistics which are of particular interest in the field of quantitative risk management. Assuming a sequential testing framework in which data is collected in sequence, where a user may choose to accept or reject a hypothesis at any point in time, we provide powerful distribution-free and anytime-valid testing methods which rely on controlled test supermartingales. Leveraging tools from online convex optimization, we show that tests can be optimized to improve their statistical power, with asymptotic guarantees for rejecting false hypotheses. By "inverting the test", these methods are extended to the task of confidence sequence building. Lastly, we implement these techniques on a range of simple examples to demonstrate their effectiveness.