Stochastische Optimierung befasst sich im weitesten Sinne mit Optimierungsproblemen, die von Zufallparametern in der Zielfunktion oder den Restriktionen beeinflusst werden. Die Lösung solcher Probleme zielt in der Regel darauf ab, kostenoptimale Entscheidungen zu treffen, die zugleich robust gegenüber der Wirkungs des Zufalls sind. Eine typische Problemklasse wird durch sogenannte Wahrscheinlichkeitsrestriktionen definiert. Hierbei garantieren die Entscheidungen, dass ein zufälliges Ungleichungssystem (z.B. die Deckung des zufälligen Bedarfs einer Ware) mit einer vorgegebenen Mindestwahrscheinlichkeit eingehalten wird. Eine typische Anwendung ist die Steuerung eines Wasserreservoirs mit zufälligen Zuflüssen unter Einhaltung von Füllstandsschranken (s. Bild links). Im Kontext der Optimierung von Gasnetzwerken mit Zufallslasten spielt die Maximierung freier Kapazitäten in den Knoten bei Lastdeckung mit gegebener Wahrscheinlichkeit eine wichtige Rolle (s. Bild rechts). Die mathematische Herausforderung dieser Restriktionen besteht darin, dass die zugrundeliegenden Wahrscheinlichkeitsfunktionen nicht in expliziter Form gegeben sind, sondern nur mit begrenzter Genauigkeit numerisch approximiert werden können. Dies erschwert insbesondere die Herleitung wichtiger struktureller Eigenschaften wie Konvexität oder Differenzierbarkeit. Ein Schwerpunkt der Forschung am WIAS ist die Herleitung von Gradientenformeln für Wahrscheinlichkeitsrestriktionen sowie die Entwicklung darauf basierender algorithmischer Lösungszugänge.  In einer weiteren typischen Problemklasse ist die Zielfunktion als Erwartungswert einer von Zufallsparametern abhängigen Funktion gegeben. Das Ziel besteht darin, einen Algorithmus zu entwickeln, der mit hoher Wahrscheinlichkeit eine gute Approximation für das Minimum dieser Zielfunktion liefert. Die wesentliche mathematische Herausforderung besteht darin, nicht-asymptotische Konvergenzraten für den vorgeschlagenen Algorithmus herzuleiten. Solche für stochastische Optimierungsprobleme entwickelten Algorithmen erweisen sich als effizient bei der Lösung komplexer deterministischer Probleme. Die Idee hinter dem Zugang ist die sogenannte "Randomisierung". Eine deterministische Zielfunktion wird als Erwartungswert einer einfachen Zufallsfunktion dargestellt. Dann wird ein stochastischer Optimierungsalgorithmus mit deutlich billigeren Iterationen benutzt, um das deterministische Problem mit hoher Wahrscheinlichkeit zu lösen.

## Publikationen

### Monografien

• A. Bayandina, P. Dvurechensky, A. Gasnikov, Chapter 8: Mirror Descent and Convex Optimization Problems with Non-smooth Inequality Constraints, in: Large Scale and Distributed Optimization, P. Giselsson, A. Rantzer, eds., Lecture Notes in Mathematics 2227, Springer Nature Switzerland AG, Cham, 2018, pp. 181--215, (Chapter Published), DOI 10.1007/978-3-319-97478-1_8 .

• L. Ghezzi, D. Hömberg, Ch. Landry, eds., Math for the Digital Factory, 27 of Mathematics in Industry / The European Consortium for Mathematics in Industry, Springer International Publishing AG, Cham, 2017, x+348 pages, (Collection Published), DOI 10.1007/978-3-319-63957-4 .

• H. Heitsch, R. Henrion, H. Leövey, R. Mirkov, A. Möller, W. Römisch, I. Wegner-Specht, Chapter 13: Empirical Observations and Statistical Analysis of Gas Demand Data, in: Evaluating Gas Network Capacities, Th. Koch, B. Hiller, M.E. Pfetsch, L. Schewe, eds., MOS-SIAM Series on Optimization, SIAM, Philadelphia, 2015, pp. 273--290, (Chapter Published).

• B. Hiller, Ch. Hayn, H. Heitsch, R. Henrion, H. Leövey, A. Möller, W. Römisch, Chapter 14: Methods for Verifying Booked Capacities, in: Evaluating Gas Network Capacities, Th. Koch, B. Hiller, M.E. Pfetsch, L. Schewe, eds., MOS-SIAM Series on Optimization, SIAM, Philadelphia, 2015, pp. 291--315, (Chapter Published).

• P. Deuflhard, M. Grötschel, D. Hömberg, U. Horst, J. Kramer, V. Mehrmann, K. Polthier, F. Schmidt, Ch. Schütte, M. Skutella, J. Sprekels, eds., MATHEON -- Mathematics for Key Technologies, 1 of EMS Series in Industrial and Applied Mathematics, European Mathematical Society Publishing House, Zurich, 2014, 453 pages, (Collection Published).

### Artikel in Referierten Journalen

• H. Heitsch, On probabilistic capacity maximization in a stationary gas network, Optimization. A Journal of Mathematical Programming and Operations Research, (2019), published online on 10.06.2019, DOI 10.1080/02331934.2019.1625353 .
Abstract
The question for the capacity of a given gas network, i.e., determining the maximal amount of gas that can be transported by a given network, appears as an essential question that network operators and political administrations are regularly faced with. In that context we present a novel mathematical approach to assist gas network operators in managing uncertainty with respect to the demand and in exposing free network capacities while increasing reliability of transmission and supply. The approach is based on the rigorous examination of optimization problems with nonlinear probabilistic constraints. As consequence we deal with solving an optimization problem with joint probabilistic constraints over an infinite system of random inequalities. We will show that the inequality system can be reduced to a finite one in the situation of considering a tree network topology. A detailed study of the problem of maximizing free booked capacities in a stationary gas network is presented that comes up with an algebraic model involving Kirchhoff's first and second laws. The focus will be on both the theoretical and numerical side. We are going to validate a kind of rank two constraint qualification implying the differentiability of the considered capacity problem. At the numerical side we are going to solve the problem using a projected gradient decent method, where the function and gradient evaluations of the probabilistic constraints are performed by the approach of spheric-radial decomposition applied for multivariate Gaussian random variables and more general distributions.

• D.R. Baimurzina, A. Gasnikov, E.V. Gasnikova, P.E. Dvurechensky, E.I. Ershov, M.B. Kubentaeva, A.A. Lagunovskaya, Universal method of searching for equilibria and stochastic equilibria in transportation networks, Computational Mathematics and Mathematical Physics, 59 (2019), pp. 19--33.

• E.A. Vorontsova, A. Gasnikov, E.A. Gorbunov, P. Dvurechensky, Accelerated gradient-free optimization methods with a non-Euclidean proximal operator, Automation and Remote Control, 80 (2019), pp. 1487--1501.

• A. Gasnikov, P. Dvurechensky, F. Stonyakin, A.A. Titov, An adaptive proximal method for variational inequalities, Computational Mathematics and Mathematical Physics, 59 (2019), pp. 836--841.

• S. Guminov, Y. Nesterov, P. Dvurechensky, A. Gasnikov, Accelerated primal-dual gradient descent with linesearch for convex, nonconvex, and nonsmooth optimization problems, Doklady Mathematics. Maik Nauka/Interperiodica Publishing, Moscow. English. Translation of the Mathematics Section of: Doklady Akademii Nauk. (Formerly: Russian Academy of Sciences. Doklady. Mathematics)., 99 (2019), pp. 125--128.

• W. VAN Ackooij, R. Henrion, P. Pérez-Aros, Generalized gradients for probabilistic/robust (probust) constraints, Optimization. A Journal of Mathematical Programming and Operations Research, published online on 14.02.2019, DOI 10.1080/02331934.2019.1576670 .
Abstract
Probability functions are a powerful modelling tool when seeking to account for uncertainty in optimization problems. In practice, such uncertainty may result from different sources for which unequal information is available. A convenient combination with ideas from robust optimization then leads to probust functions, i.e., probability functions acting on generalized semi-infinite inequality systems. In this paper we employ the powerful variational tools developed by Boris Mordukhovich to study generalized differentiation of such probust functions. We also provide explicit outer estimates of the generalized subdifferentials in terms of nominal data.

• M. Eigel, J. Neumann, R. Schneider, S. Wolf, Risk averse stochastic structural topology optimization, Computer Methods in Applied Mechanics and Engineering, 334 (2018), pp. 470--482, DOI 10.1016/j.cma.2018.02.003 .
Abstract
A novel approach for risk-averse structural topology optimization under uncertainties is presented which takes into account random material properties and random forces. For the distribution of material, a phase field approach is employed which allows for arbitrary topological changes during optimization. The state equation is assumed to be a high-dimensional PDE parametrized in a (finite) set of random variables. For the examined case, linearized elasticity with a parametric elasticity tensor is used. Instead of an optimization with respect to the expectation of the involved random fields, for practical purposes it is important to design structures which are also robust in case of events that are not the most frequent. As a common risk-aware measure, the Conditional Value at Risk (CVaR) is used in the cost functional during the minimization procedure. Since the treatment of such high-dimensional problems is a numerically challenging task, a representation in the modern hierarchical tensor train format is proposed. In order to obtain this highly efficient representation of the solution of the random state equation, a tensor completion algorithm is employed which only required the pointwise evaluation of solution realizations. The new method is illustrated with numerical examples and compared with a classical Monte Carlo sampling approach.

• L. Adam, M. Branda, H. Heitsch, R. Henrion, Solving joint chance constrained problems using regularization and Benders' decomposition, Annals of Operations Research, pp. published online on 08.11.2018, urlhttps://doi.org/10.1007/s10479-018-3091-9, DOI 10.1007/s10479-018-3091-9 .
Abstract
In this paper we investigate stochastic programms with joint chance constraints. We consider discrete scenario set and reformulate the problem by adding auxiliary variables. Since the resulting problem has a difficult feasible set, we regularize it. To decrease the dependence on the scenario number, we propose a numerical method by iteratively solving a master problem while adding Benders cuts. We find the solution of the slave problem (generating the Benders cuts) in a closed form and propose a heuristic method to decrease the number of cuts. We perform a numerical study by increasing the number of scenarios and compare our solution with a solution obtained by solving the same problem with continuous distribution.

• A. Gasnikov, P. Dvurechensky, M. Zhukovskii, S. Kim, S. Plaunov, D. Smirnov, F. Noskov, About the power law of the PageRank vector distribution. Part 2. Backley--Osthus model, power law verification for this model and setup of real search engines, Numerical Analysis and Applications, 11 (2018), pp. 16--32, DOI 10.1134/S1995423918010032 .

• A. Hantoute, R. Henrion, P. Pérez-Aros, Subdifferential characterization of probability functions under Gaussian distribution, Mathematical Programming. A Publication of the Mathematical Programming Society, 174 (2019), pp. 167--194 (published online on 29.01.2018), DOI 10.1007/s10107-018-1237-9 .
Abstract
Probability functions figure prominently in optimization problems of engineering. They may be nonsmooth even if all input data are smooth. This fact motivates the consideration of subdifferentials for such typically just continuous functions. The aim of this paper is to provide subdifferential formulae of such functions in the case of Gaussian distributions for possibly infinite-dimensional decision variables and nonsmooth (locally Lipschitzian) input data. These formulae are based on the spheric-radial decomposition of Gaussian random vectors on the one hand and on a cone of directions of moderate growth on the other. By successively adding additional hypotheses, conditions are satisfied under which the probability function is locally Lipschitzian or even differentiable.

• P. Dvurechensky, A. Gasnikov, A. Lagunovskaya, Parallel algorithms and probability of large deviation for stochastic convex optimization problems, Numerical Analysis and Applications, 11 (2018), pp. 33--37, DOI 10.1134/S1995423918010044 .

• R. Henrion, W. Römisch, Problem-based optimal scenario generation and reduction in stochastic programming, Mathematical Programming. A Publication of the Mathematical Programming Society, pp. published online on 04.10.2018, urlhttps://doi.org/10.1007/s10107-018-1337-6, DOI 10.1007/s10107-018-1337-6 .
Abstract
Scenarios are indispensable ingredients for the numerical solution of stochastic programs. Earlier approaches to optimal scenario generation and reduction are based on stability arguments involving distances of probability measures. In this paper we review those ideas and suggest to make use of stability estimates based only on problem specific data. For linear two-stage stochastic programs we show that the problem-based approach to optimal scenario generation can be reformulated as best approximation problem for the expected recourse function which in turn can be rewritten as a generalized semi-infinite program. We show that the latter is convex if either right-hand sides or costs are random and can be transformed into a semi-infinite program in a number of cases. We also consider problem-based optimal scenario reduction for two-stage models and optimal scenario generation for chance constrained programs. Finally, we discuss problem-based scenario generation for the classical newsvendor problem.

• T. González Grandón, H. Heitsch, R. Henrion, A joint model of probabilistic/robust constraints for gas transport management in stationary networks, Computational Management Science, 14 (2017), pp. 443--460, DOI 10.1007/s10287-017-0284-7 .
Abstract
We present a novel mathematical algorithm to assist gas network operators in managing uncertainty, while increasing reliability of transmission and supply. As a result, we solve an optimization problem with a joint probabilistic constraint over an infinite system of random inequalities. Such models arise in the presence of uncertain parameters having partially stochastic and partially non-stochastic character. The application that drives this new approach is a stationary network with uncertain demand (which are stochastic due to the possibility of fitting statistical distributions based on historical measurements) and with uncertain roughness coefficients in the pipes (which are uncertain but non-stochastic due to a lack of attainable measurements). We study the sensitivity of local uncertainties in the roughness coefficients and their impact on a highly reliable network operation. In particular, we are going to answer the question, what is the maximum uncertainty that is allowed (shaping a 'maximal' uncertainty set) around nominal roughness coefficients, such that random demands in a stationary gas network can be satisfied at given high probability level for no matter which realization of true roughness coefficients within the uncertainty set. One ends up with a constraint, which is probabilistic with respect to the load of gas and robust with respect to the roughness coefficients. We demonstrate how such constraints can be dealt with in the framework of the so-called spheric-radial decomposition of multivariate Gaussian distributions. The numerical solution of a corresponding optimization problem is illustrated. The results might assist the network operator with the implementation of cost-intensive roughness measurements.

• A.L. Diniz, R. Henrion, On probabilistic constraints with multivariate truncated Gaussian and lognormal distributions, Energy Systems, 8 (2017), pp. 149--167, DOI 10.1007/s12667-015-0180-6 .

• A. Gasnikov, E. Gasnikova, P. Dvurechensky, A. Mohammed, E. Chernousova, About the power law of the PageRank vector component distribution. Part 1. Numerical methods for finding the PageRank vector (Original Russian text published in Sib. Zh. Vychisl. Mat., 20 (2017), pp. 359--378), Numerical Analysis and Applications, 10 (2017), pp. 299--312.

• V. Guigues, R. Henrion, Joint dynamic probabilistic constraints with projected linear decision rules, Optimization Methods & Software, 32 (2017), pp. 1006--1032.
Abstract
We consider multistage stochastic linear optimization problems combining joint dynamic probabilistic constraints with hard constraints. We develop a method for projecting decision rules onto hard constraints of wait-and-see type. We establish the relation between the original (infinite dimensional) problem and approximating problems working with projections from different subclasses of decision policies. Considering the subclass of linear decision rules and a generalized linear model for the underlying stochastic process with noises that are Gaussian or truncated Gaussian, we show that the value and gradient of the objective and constraint functions of the approximating problems can be computed analytically.

• W. VAN Ackooij, R. Henrion, (Sub-) Gradient formulae for probability functions of random inequality systems under Gaussian distribution, SIAM/ASA Journal on Uncertainty Quantification, 5 (2017), pp. 63--87, DOI 10.1137/16M1061308 .
Abstract
We consider probability functions of parameter-dependent random inequality systems under Gaussian distribution. As a main result, we provide an upper estimate for the Clarke subdifferential of such probability functions without imposing compactness conditions. A constraint qualification ensuring continuous differentiability is formulated. Explicit formulae are derived from the general result in case of linear random inequality systems. In the case of a constant coefficient matrix an upper estimate for even the smaller Mordukhovich subdifferential is proven.

• H. Heitsch, H. Leövey, W. Römisch, Are quasi-Monte Carlo algorithms efficient for two-stage stochastic programs?, Computational Optimization and Applications. An International Journal, 65 (2016), pp. 567--603.
Abstract
Quasi-Monte Carlo algorithms are studied for designing discrete approximations of two-stage linear stochastic programs with random right-hand side and continuous probability distribution. The latter should allow for a transformation to a distribution with independent marginals. The two-stage integrands are piecewise linear, but neither smooth nor lie in the function spaces considered for QMC error analysis. We show that under some weak geometric condition on the two-stage model all terms of their ANOVA decomposition, except the one of highest order, are continuously differentiable and that first and second order ANOVA terms have mixed first order partial derivatives. Hence, randomly shifted lattice rules (SLR) may achieve the optimal rate of convergence not depending on the dimension if the effective superposition dimension is at most two. We discuss effective dimensions and dimension reduction for two-stage integrands. The geometric condition is shown to be satisfied almost everywhere if the underlying probability distribution is normal and principal component analysis (PCA) is used for transforming the covariance matrix. Numerical experiments for a large scale two-stage stochastic production planning model with normal demand show that indeed convergence rates close to the optimal are achieved when using SLR and randomly scrambled Sobol' point sets accompanied with PCA for dimension reduction.

• R. Hildebrand, Spectrahedral cones generated by rank 1 matrices, Journal of Global Optimization. An International Journal Dealing with Theoretical and Computational Aspects of Seeking Global Optima and Their Applications in Science, Management and Engineering, 64 (2016), pp. 349--397.

• C. Gotzes, H. Heitsch, R. Henrion, R. Schultz, On the quantification of nomination feasibility in stationary gas networks with random load, Mathematical Methods of Operations Research, 84 (2016), pp. 427--457.
Abstract
The paper considers the computation of the probability of feasible load constellations in a stationary gas network with uncertain demand. More precisely, a network with a single entry and several exits with uncertain loads is studied. Feasibility of a load constellation is understood in the sense of an existing flow meeting these loads along with given pressure bounds in the pipes. In a first step, feasibility of deterministic exit loads is characterized algebraically and these general conditions are specified to networks involving at most one cycle. This prerequisite is essential for determining probabilities in a stochastic setting when exit loads are assumed to follow some (joint) Gaussian distribution when modeling uncertain customer demand. The key of our approach is the application of the spheric-radial decomposition of Gaussian random vectors coupled with Quasi Monte-Carlo sampling. This approach requires an efficient algorithmic treatment of the mentioned algebraic relations moreover depending on a scalar parameter. Numerical results are illustrated for different network examples and demonstrate a clear superiority in terms of precision over simple generic Monte-Carlo sampling. They lead to fairly accurate probability values even for moderate sample size.

• A.V. Gasnikov, P. Dvurechensky, Y.E. Nesterov, Stochastic gradient methods with inexact oracle, Proceedings of Moscow Institute of Physics and Technology, 8:1 (2016), pp. 41--91.

• A. Gasnikov, P. Dvurechensky, I. Usmanova, On accelerated randomized methods, Proceedings of Moscow Institute of Physics and Technology, 8:2 (2016), pp. 67--100.

• A. Gasnikov, P. Dvurechensky, Stochastic intermediate gradient method for convex optimization problems, Doklady Mathematics. Maik Nauka/Interperiodica Publishing, Moscow. English. Translation of the Mathematics Section of: Doklady Akademii Nauk. (Formerly: Russian Academy of Sciences. Doklady. Mathematics)., 93 (2016), pp. 148--151.

• P. Dvurechensky, A. Gasnikov, Stochastic intermediate gradient method for convex problems with inexact stochastic oracle, Journal of Optimization Theory and Applications, 171 (2016), pp. 121--145.

• A. Gasnikov, E. Gasnikova, P. Dvurechensky, E. Ershov, A. Lagunovskaia, Searching for the stochastic equilibria in the transport models of equilibrium flow distribution (in Russian), Proceedings of Moscow Institute of Physics and Technology, 7 (2015), pp. 114--128.

• A. Gasnikov, P. Dvurechensky, D. Kamzolov, Y. Nesterov, V. Spokoiny, P. Stetsyuk, A. Suvorikova, A. Chernov, Searching for equilibriums in multistage transport models (in Russian), Proceedings of Moscow Institute of Physics and Technology, 7 (2015), pp. 143--155.

• I. Bremer, R. Henrion, A. Möller, Probabilistic constraints via SQP solver: Application to a renewable energy management problem, Computational Management Science, 12 (2015), pp. 435--459.
Abstract
The aim of this paper is to illustrate the efficient solution of nonlinear optimization problems with joint probabilistic constraints by means of an SQP method. Here, the random vector is assumed to obey some multivariate Gaussian distribution. The numerical solution approach is applied to a renewable energy management problem. We consider a coupled system of hydro and wind power production used in order to satisfy some local demand of energy and to sell/buy excessive or missing energy on a day-ahead and intraday market, respectively. A short term planning horizon of 2 days is considered and only wind power is assumed to be random. In the first part of the paper, we develop an appropriate optimization problem involving a probabilistic constraint reflecting demand satisfaction. Major attention will be payed to formulate this probabilistic constraint not directly in terms of random wind energy produced but rather in terms of random wind speed, in order to benefit from a large data base for identifying an appropriate distribution of the random parameter. The second part presents some details on integrating Genz' code for Gaussian probabilities of rectangles into the environment of the SQP solver SNOPT. The procedure is validated by means of a simplified optimization problem which by its convex structure allows to estimate the gap between the numerical and theoretical optimal values, respectively. In the last part, numerical results are presented and discussed for the original (nonconvex) optimization problem.

• TH. Arnold, R. Henrion, A. Möller, S. Vigerske, A mixed-integer stochastic nonlinear optimization problem with joint probabilistic constraints, Pacific Journal of Optimization. An International Journal, 10 (2014), pp. 5--20.
Abstract
We illustrate the solution of a mixed-integer stochastic nonlinear optimization problem in an application of power management. In this application, a coupled system consisting of a hydro power station and a wind farm is considered. The objective is to satisfy the local energy demand and sell any surplus energy on a spot market for a short time horizon. Generation of wind energy is assumed to be random, so that demand satisfaction is modeled by a joint probabilistic constraint taking into account the multivariate distribution. The turbine is forced to either operate between given positive limits or to be shut down. This introduces additional binary decisions. The numerical solution procedure is presented and results are illustrated.

• K. Emich, R. Henrion, W. Römisch, Conditioning of linear-quadratic two-stage stochastic optimization problems, Mathematical Programming. A Publication of the Mathematical Programming Society, 148 (2014), pp. 201--221.
Abstract
In this paper a condition number for linear-quadratic two-stage stochastic optimization problems is introduced as the Lipschitz modulus of the multifunction assigning to a (discrete) probability distribution the solution set of the problem. Being the outer norm of the Mordukhovich coderivative of this multifunction, the condition number can be estimated from above explicitly in terms of the problem data by applying appropriate calculus rules. Here, a chain rule for the extended partial second-order subdifferential recently proved by Mordukhovich and Rockafellar plays a crucial role. The obtained results are illustrated for the example of two-stage stochastic optimization problems with simple recourse.

• W. VAN Ackooij, R. Zorgati, R. Henrion, A. Möller, Joint chance constrained programming for hydro reservoir management, Optimization and Engineering. International Multidisciplinary Journal to Promote Optimization Theory & Applications in Engineering Sciences, 15 (2014), pp. 509--531.

• W. VAN Ackooij, R. Henrion, Gradient formulae for nonlinear probabilistic constraints with Gaussian and Gaussian-like distributions, SIAM Journal on Optimization, 24 (2014), pp. 1864--1889.
Abstract
Probabilistic constraints represent a major model of stochastic optimization. A possible approach for solving probabilistically constrained optimization problems consists in applying nonlinear programming methods. In order to do so, one has to provide sufficiently precise approximations for values and gradients of probability functions. For linear probabilistic constraints under Gaussian distribution this can be successfully done by analytically reducing these values and gradients to values of Gaussian distribution functions and computing the latter, for instance, by Genz' code. For nonlinear models one may fall back on the spherical-radial decomposition of Gaussian random vectors and apply, for instance, Deák's sampling scheme for the uniform distribution on the sphere in order to compute values of corresponding probability functions. The present paper demonstrates how the same sampling scheme can be used in order to simultaneously compute gradients of these probability functions. More precisely, we prove a formula representing these gradients in the Gaussian case as a certain integral over the sphere again. Later, the result is extended to alternative distributions with an emphasis on the multivariate Student (or T-) distribution.

### Beiträge zu Sammelwerken

• F. Stonyakin, D. Dvinskikh, P. Dvurechensky, A. Kroshnin, O. Kuznetsova, A. Agafonov, A. Gasnikov, A. Tyurin, C.A. Uribe, D. Pasechnyuk, S. Artamonov, Gradient methods for problems with inexact model of the objective, in: Proceedings of the 18th International Conference on Mathematical Optimization Theory and Operations Research (MOTOR 2019), M. Khachay, Y. Kochetov, P. Pardalos, eds., 11548 of Lecture Notes in Computer Science, Springer Nature Switzerland AG 2019, Cham, Switzerland, 2019, pp. 97--114, DOI 10.1007/978-3-030-22629-9_8 .
Abstract
We consider optimization methods for convex minimization problems under inexact information on the objective function. We introduce inexact model of the objective, which as a particular cases includes inexact oracle  and relative smoothness condition . We analyze gradient method which uses this inexact model and obtain convergence rates for convex and strongly convex problems. To show potential applications of our general framework we consider three particular problems. The first one is clustering by electorial model introduced in . The second one is approximating optimal transport distance, for which we propose a Proximal Sinkhorn algorithm. The third one is devoted to approximating optimal transport barycenter and we propose a Proximal Iterative Bregman Projections algorithm. We also illustrate the practical performance of our algorithms by numerical experiments.

• P. Dvurechensky, D. Dvinskikh, A. Gasnikov, C.A. Uribe, A. Nedić, Decentralize and randomize: Faster algorithm for Wasserstein barycenters, in: Advances in Neural Information Processing Systems 31, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett, eds., Curran Associates, Inc., 2018, pp. 10760--10770.

• TH. Arnold, R. Henrion, M. Grötschel, W. Römisch ET AL., B4 -- A Jack of all trades? Solving stochastic mixed-integer nonlinear constraint programs, in: MATHEON -- Mathematics for Key Technologies, M. Grötschel, D. Hömberg, J. Sprekels, V. Mehrmann ET AL., eds., 1 of EMS Series in Industrial and Applied Mathematics, European Mathematical Society Publishing House, Zurich, 2014, pp. 135--146.

### Preprints, Reports, Technical Reports

• D. Gahururu, M. Hintermüller, S.-M. Stengl, Th.M. Surowiec, Generalized Nash equilibrium problems with partial differential operators: Theory, algorithms, and risk aversion, Preprint no. 2654, WIAS, Berlin, 2019, DOI 10.20347/WIAS.PREPRINT.2654 .
Abstract, PDF (337 kByte)
PDE-constrained (generalized) Nash equilibrium problems (GNEPs) are considered in a deterministic setting as well as under uncertainty. This includes a study of deterministic GNEPs with nonlinear and/or multivalued operator equations as forward problems and PDE-constrained GNEPs with uncertain data. The deterministic nonlinear problems are analyzed using the theory of generalized convexity for set-valued operators, and a variational approximation approach is proposed. The stochastic setting includes a detailed overview of the recently developed theory and algorithms for risk-averse PDE-constrained optimization problems. These new results open the way to a rigorous study of stochastic PDE-constrained GNEPs.

• CH. Bayer, R.F. Tempone , S. Wolfers, Pricing American options by exercise rate optimization, Preprint no. 2651, WIAS, Berlin, 2019, DOI 10.20347/WIAS.PREPRINT.2651 .
Abstract, PDF (761 kByte)
We present a novel method for the numerical pricing of American options based on Monte Carlo simulation and the optimization of exercise strategies. Previous solutions to this problem either explicitly or implicitly determine so-called optimal exercise regions, which consist of points in time and space at which a given option is exercised. In contrast, our method determines the exercise rates of randomized exercise strategies. We show that the supremum of the corresponding stochastic optimization problem provides the correct option price. By integrating analytically over the random exercise decision, we obtain an objective function that is differentiable with respect to perturbations of the exercise rate even for finitely many sample paths. The global optimum of this function can be approached gradually when starting from a constant exercise rate. Numerical experiments on vanilla put options in the multivariate Black-Scholes model and a preliminary theoretical analysis underline the efficiency of our method, both with respect to the number of time-discretization steps and the required number of degrees of freedom in the parametrization of the exercise rates. Finally, we demonstrate the flexibility of our method through numerical experiments on max call options in the classical Black-Scholes model, and vanilla put options in both the Heston model and the non-Markovian rough Bergomi model.

• M.H. Farshbaf Shaker, M. Gugat, H. Heitsch, R. Henrion, Optimal Neumann boundary control of a vibrating string with uncertain initial data and probabilistic terminal constraints, Preprint no. 2626, WIAS, Berlin, 2019, DOI 10.20347/WIAS.PREPRINT.2626 .
Abstract, PDF (424 kByte)
In optimal control problems, often initial data are required that are not known exactly in practice. In order to take into account this uncertainty, we consider optimal control problems for a system with an uncertain initial state. A finite terminal time is given. On account of the uncertainty of the initial state, it is not possible to prescribe an exact terminal state. Instead, we are looking for controls that steer the system into a given neighborhood of the desired terminal state with sufficiently high probability. This neighborhood is described in terms of an inequality for the terminal energy. The probabilistic constraint in the considered optimal control problem leads to optimal controls that are robust against the inevitable uncertainties of the initial state. We show the existence of such optimal controls. Numerical examples with optimal Neumann control of the wave equation are presented.

• D. Adelhütte, D. Assmann, T. González Grandón, M. Gugat, H. Heitsch, R. Henrion, F. Liers, S. Nitsche, R. Schultz, M. Stingl, D. Wintergerst, Joint model of probabilistic-robust (probust) constraints with application to gas network optimization, Preprint no. 2550, WIAS, Berlin, 2018, DOI 10.20347/WIAS.PREPRINT.2550 .
Abstract, PDF (8736 kByte)
Optimization problems under uncertain conditions abound in many real-life applications. While solution approaches for probabilistic constraints are often developed in case the uncertainties can be assumed to follow a certain probability distribution, robust approaches are usually applied in case solutions are sought that are feasible for all realizations of uncertainties within some predefined uncertainty set. As many applications contain different types of uncertainties that require robust as well as probabilistic treatments, we introduce a class of joint probabilistic/robust constraints. Focusing on complex uncertain gas network optimization problems, we show the relevance of this class of problems for the task of maximizing free booked capacities in an algebraic model for a stationary gas network. We furthermore present approaches for finding their solution. Finally, we study the problem of controlling a transient system that is governed by the wave equation. The task consists in determining controls such that a certain robustness measure remains below some given upper bound with high probability.

### Vorträge, Poster

• H. Heitsch, On probabilistic capacity maximization in a stationary gas network, International Congress on Industrial and Applied Mathematics (ICIAM), Minisymposium ``Mathematical Optimization and Gas Transport Networks: Academic Developments II'', July 15 - 19, 2019, Valencia, Spain, July 16, 2019.

• H. Heitsch, Optimal Neumann boundary control of the vibrating string with uncertain initial data and probabilistic terminal constraints, The XV International Conference on Stochastic Programming (ICSP XV), Minisymposium ``Nonlinear Programming with Probability Functions'', July 29 - August 2, 2019, Norwegian University of Science and Technology, Trondheim, Norway, July 30, 2019.

• CH. Bayer, Pricing american options by exercise rate optimization, Workshop on Financial Risks and Their Management, February 19 - 20, 2019, Ryukoku University, Wagenkan, Kyoto, Japan, February 19, 2019.

• R. Henrion, Chance constraints then and now, International Conference on Stochastic Optimization and Related Topics, April 25 - 26, 2019, Mühlheim an der Ruhr, April 26, 2019.

• R. Henrion, Nonsmoothness in the context of probability functions, Workshop 4 within the Special Semester on Optimization ``Nonsmooth Optimization'', November 25 - 27, 2019, Johann Radon Institute for Computational and Applied Mathematics (RICAM), Linz, Austria, November 25, 2019.

• R. Henrion, On derivatives of probability functions, Workshop ``Statistics, Risk & Optimization'', Universität Wien, Austria, September 27, 2019.

• R. Henrion, On some extended models of chance constraints, Workshop ``Mathematical Optimization of Systems Impacted by Rare, High-Impact Random Events'', June 24 - 28, 2019, Institute for Computational and Experimental Research in Mathematics (ICERM), Providence, USA, June 24, 2019.

• R. Henrion, Optimal Neumann boundary control of the vibrating string under random initial conditions, OVA9: 9th International Seminar on Optimization and Variational Analysis, Universidad Miguel Hernández, Elche, Spain, September 2, 2019.

• R. Henrion, Optimal Neumann boundary control of the vibrating string with uncertain initial data, ICCOPT 2019 -- Sixth International Conference on Continuous Optimization, Session ``PDE-constrained Optimization under Uncertainty (Part I)'', August 5 - 8, 2019, Berlin, August 8, 2019.

• R. Henrion, Optimal probabilistic control of the vibrating string under random initial conditions, PGMO DAYS 2018, Session 1E ``Stochastic Optimal Control'', December 3 - 4, 2019, Gaspard Monge Program for Optimization, Operations Research and their Interaction with Data Science, EDF'Lab Paris-Saclay, Palaiseau, France, December 4, 2019.

• R. Henrion, Optimization problems with probust constraints: Theory, applications and algorithmic solution, XXXVIII Spanish Conference on Statistics and Operations Research, September 3 - 6, 2019, Universitat Politècnica de València, Alcoi, Spain, September 5, 2019.

• R. Henrion, Probabilistic constraints in optimization with PDEs, Workshop 3 within the Special Semester on Optimization ``Optimization and Inversion under Uncertainty'', November 11 - 15, 2019, Johann Radon Institute for Computational and Applied Mathematics (RICAM), Linz, Austria, November 13, 2019.

• R. Henrion, Problèmes d'optimisation sous contraintes en probabilité, Spring School in Nonsmooth Analysis and Optimization, April 16 - 18, 2019, Université Mohammed V, Rabat, Morocco.

• R. Henrion, Robust control of a sweeping process with probabilistic end-point constraints, The XV International Conference on Stochastic Programming (ICSP XV), Minisymposium ``Nonlinear Programming with Probability Functions'', July 29 - August 2, 2019, Norwegian University of Science and Technology, Trondheim, Norway, July 30, 2019.

• R. Henrion, Robust control of a sweeping process with probabilistic end-point constraints, Workshop ``Nonsmooth and Variational Analysis'', January 28 - February 1, 2019, Erwin Schrödinger International Institute for Mathematics and Physics, Vienna, Austria, January 31, 2019.

• H. Heitsch, A probabilistic approach to optimization in gas transport, 2nd Conference on Mathematics of Gas Transport (MoG-2), October 10 - 11, 2018, Konrad-Zuse-Zentrum für Informationstechnik Berlin, October 10, 2018.

• D. Dvinskikh, Decentralize and randomize: Faster algorithm for Wasserstein barycenters, Thirty-second Conference on Neural Information Processing Systems, December 3 - 8, 2018, Montréal, Canada, December 6, 2018.

• P. Dvurechensky, D. Dvinskikh, Decentralize and randomize: Faster algorithm for Wasserstein barycenters, Thirty-Second Conference On Neural Information Processing Systems, Montréal, Canada, December 3 - 8, 2018.

• P. Dvurechensky, Decentralize and randomize: Faster algorithm for Wasserstein barycenters, Thirty-second Conference on Neural Information Processing Systems, December 3 - 8, 2018, Montréal, Canada, December 6, 2018.

• R. Henrion, Dynamic chance constraints under continuous random distribution, PGMO DAYS 2018, Session 1E ``Stochastic Optimization'', November 20 - 21, 2018, Gaspard Monge Program for Optimization, Operations Research and their Interaction with Data Science, EDF'Lab Paris-Saclay, Palaiseau, France, November 21, 2018.

• R. Henrion, Maximization of free capacities in gas networks with random load, Conference ``Variational Analysis -- Challenges in Energy'', June 4 - 6, 2018, Castro Urdiales, Spain, June 4, 2018.

• R. Henrion, Optimization problems under probabilistic constraints, 3rd Russian-German Conference on MultiScale BioMathematics: Coherent Modeling of Human Body System, November 7 - 9, 2018, Lomonosov Moscow State University, Russian Federation, November 8, 2018.

• R. Henrion, Optimization problems with probust constraints, Colloquium & International Conference on Variational Analysis and Nonsmooth Optimization, June 28 - July 1, 2018, Martin-Luther-Universität Halle-Wittenberg, Halle, June 28, 2018.

• R. Henrion, Perspectives in probabilistic programming under continuous random distributions, Workshop ``New Directions in Stochastic Optimisation'', August 19 - 25, 2018, Mathematisches Forschungsinstitut Oberwolfach, August 20, 2018.

• R. Henrion, Probabilistic programming with applications, Universidad Miguel Hernández de Elche, Instituto Centro de Investigación Operativa, Spain, September 13, 2018.

• M. Marschall, Bayesian inversion with adaptive low-rank approximation, Analysis, Control and Inverse Problems for PDEs -- Workshop of the French-German-Italian LIA (Laboratoire International Associe) COPDESC on Applied Analysis, November 26 - 30, 2018, University of Naples Federico II and Accademia Pontaniana, Italy, November 29, 2018.

• H. Heitsch, A probabilistic approach to optimization problems in gas transport networks, SESO 2017 International Thematic Week ``Smart Energy and Stochastic Optimization'', May 30 - June 1, 2017, ENSTA ParisTech and École des Ponts ParisTech, Paris, France, June 1, 2017.

• H. Heitsch, A probabilistic approach to optimization problems in gas transport networks, CIM-WIAS Workshop ``Topics in Applied Analysis and Optimisation'', December 6 - 8, 2017, International Center for Mathematics, University of Lisbon, Portugal, December 6, 2017.

• H. Heitsch, On probabilistic capacity maximization in stationary gas networks, 21st Conference of the International Federation of Operational Research Societies (IFORS 2017), Invited Session TB20 ``Optimization of Gas Networks 2'', July 17 - 21, 2017, Quebec, Canada, July 18, 2017.

• P. Dvurechensky, A unified view on accelerated randomized optimization methods: Coordinate descent, directional search, derivative-free method, Foundations of Computational Mathematics (FoCM 2017), Barcelona, Spain, July 17 - 19, 2017.

• R. Henrion, Contraintes en probabilité: Formules du gradient et applications, Workshop ``MAS-MODE 2017'', Institut Henri Poincaré, Paris, France, January 9, 2017.

• R. Henrion, On M-stationary condition for a simple electricity spot market model, Workshop ``Variational Analysis and Applications for Modelling of Energy Exchange'', May 4 - 5, 2017, Université Perpignan, France, May 4, 2017.

• R. Henrion, On a joint model for probabilistic/robust constraints with an application to gas networks under uncertainties, Workshop ``Models and Methods of Robust Optimization'', March 9 - 10, 2017, Fraunhofer-Institut für Techno- und Wirtschaftsmathematik ITWM, Kaiserslautern, March 10, 2017.

• R. Henrion, Optimization problems under robust constraints with applications to gas networks under uncertainty, The Eighth Australia-China Workshop on Optimization (ACWO 2017), December 4, 2017, Curtin University, Perth, Australia, December 4, 2017.

• R. Henrion, Probabilistic constraints in infinite dimensions, Universität Wien, Institut für Statistik und Operations Research, Austria, November 6, 2017.

• R. Henrion, Probabilistic constraints: Convexity issues and beyond, XII International Symposium on Generalized Convexity and Monotonicity, August 27 - September 2, 2017, Hajdúszoboszló, Hungary, August 29, 2017.

• R. Henrion, Probabilistic programming in infinite dimensions, The South Pacific Optimization Meeting in Western Australia 2017 (SPOM in WA 2017), December 8 - 10, 2017, Curtin University, Perth, Australia, December 9, 2017.

• R. Henrion, Probabilistic programming: Structural properties and applications, Control and Optimization Conference on the occasion of Frédéric Bonnans 60th birthday, November 15 - 17, 2017, Electricité de France, Palaiseau, France, November 17, 2017.

• R. Henrion, Problèmes d'optimisation sous contraintes en probabilité, Université de Bourgogne, Département de Mathématiques, Dijon, France, October 25, 2017.

• R. Henrion, Subdifferential characterization of Gaussian probability functions, SESO 2017 International Thematic Week ``Smart Energy and Stochastic Optimization'', May 30 - June 1, 2017, ENSTA ParisTech and École des Ponts ParisTech, Paris, France, June 1, 2017.

• R. Henrion, Subdifferential estimates for Gaussian probability functions, HCM Workshop: Nonsmooth Optimization and its Applications, May 15 - 19, 2017, Hausdorff Center for Mathematics, Bonn, May 17, 2017.

• R. Henrion, Subdifferential of probability functions under Gaussian distribution, The Second Pacific Optimization Conference (POC2017), December 4 - 7, 2017, Curtin University, Perth, Australia, December 6, 2017.

• H. Heitsch, Nonlinear probabilistic constraints in gas transportation problems, WIAS-PGMO Workshop on Nonsmooth and Stochastic Optimization with Applications to Energy Management, May 10 - 12, 2016, WIAS Berlin, Australia, May 11, 2016.

• H. Heitsch, Optimization in gas transport networks using nonlinear probabilistic constraints, XIV International Conference on Stochastic Programming (ICSP 2016), Thematic Session: Probabilistic Constraints: Applications and Theory, June 25 - July 1, 2016, Búzios, Brazil, June 28, 2016.

• R. Hildebrand, Canonical barriers on convex cones, Oberseminar Geometrische Analysis, Johann Wolfgang Goethe-Universität Frankfurt am Main, Fachbereich Mathematik, April 26, 2016.

• J. Neumann, The phase field approach for topology optimization under uncertainties, ZIB Computational Medicine and Numerical Mathematics Seminar, Konrad-Zuse-Zentrum für Informationstechnik Berlin, August 25, 2016.

• I. Bremer, Dealing with probabilistic constraints under multivariate normal distribution in a standard SQP solver by using Genz' method, WIAS-PGMO Workshop on Nonsmooth and Stochastic Optimization with Applications to Energy Management, May 10 - 12, 2016, WIAS Berlin, May 11, 2016.

• R. Henrion, (Sub-)Gradient formulae for Gaussian probability functions, XIV International Conference on Stochastic Programming (ICSP 2016), Thematic Session: Probabilistic Constraints: Applications and Theory, June 25 - July 1, 2016, Búzios, Brazil, June 28, 2016.

• R. Henrion, Aspects of nondifferentiability for probability functions, 7th International Seminar on Optimization and Variational Analysis, June 1 - 3, 2016, Universidad de Alicante, Spain, June 2, 2016.

• R. Henrion, Aspects of nonsmoothness for Gaussian probability functions, PGMO Days 2016 -- Gaspard Monge Program for Optimization and Operations Research, November 8 - 9, 2016, Electricité de France, Palaiseau, France, November 9, 2016.

• R. Henrion, Formules du gradient pour des fonctions probabilistes Gaussiennes, Workshop on Offshore Wind Generation, September 9, 2016, Electricité de France R&D, Paris, France, September 9, 2016.

• R. Henrion, Initiation aux problèmes d'optimisation sous contraintes en probabilité, Workshop ``Optimisation en Milieu Aléatoire'', November 8, 2016, Institut des Sciences Informatiques et de leurs Interactions, GdR 720 ISIS (Information, Signal, Image et ViSion), Paristech Télécom, Paris, France, November 8, 2016.

• R. Henrion, Optimisation sous contraintes en probabilité, Séminaire du Groupe de Travail ``Modèles Stochastiques en Finance'', Ecole Nationale Supérieure des Techniques Avancées (ENSTA) ParisTech, Palaiseau, France, November 28, 2016.

• R. Henrion, Robust-stochastic optimization problems in stationary gas networks, Conference ``Mathematics of Gas Transport'', October 6 - 7, 2016, Zuse Institut Berlin, October 6, 2016.

• H. Heitsch, Optimization of booked capacity in gas transport networks using nonlinear probabilistic constraints, 2nd International Symposium on Mathematical Programming (ISMP 2015), Cluster ``Optimization in Energy Systems'', July 13 - 17, 2015, Pittsburgh, USA, July 17, 2015.

• R. Hildebrand, Geometry of barriers for 3-dimensional cones, Optimization and Applications in Control and Data Science, May 13 - 15, 2015, Moscow Institute of Physics and Technology, PreMoLab, Moscow, Russian Federation, May 15, 2015.

• R. Hildebrand, Rank 1 generated spectrahedral cones, Frontiers of High Dimensional Statistics, Optimization, and Econometrics, February 26 - 27, 2015, Higher School of Economics, Moscow, Russian Federation, February 26, 2015.

• R. Hildebrand , Optimizing strategies in energy and storage markets, Matheon Center Days, Technische Universität Berlin, April 16, 2015.

• P. Dvurechensky, Semi-supervised pagerank model learning with gradient-free optimization methods, Traditional Youth School ``Control, Information and Optimization'', June 14 - 20, 2015, Moscow, Russian Federation, June 17, 2015.

• P. Dvurechensky, Stochastic intermediate gradient method: Convex and strongly convex cases, Information Technologies and Systems 2015, September 6 - 11, 2015, Russian Academy of Sciences, Institute for Information Transmission Problems, Sochi, Russian Federation, September 9, 2015.

• R. Henrion, (Sub-) Gradient formulae for probability functions with Gaussian distribution, PGMO Days 2015 -- Gaspard Monge Program for Optimization and Operations Research, October 27 - 28, 2015, ENSTA ParisTech, Palaiseau, France, October 28, 2015.

• R. Henrion, (Sub-)Gradient formulae for probability functions with applications to power management, Universidad de Chile, Centro de Modelamiento Matemático, Santiago de Chile, Chile, November 25, 2015.

• R. Henrion, Conditioning of linear-quadratic two-stage stochastic optimization problems, Charles University, Faculty of Mathematics and Physics, Prague, Czech Republic, March 26, 2015.

• R. Henrion, On some relations between probability functions and variational analysis, International Workshop ``Variational Analysis and Applications'', August 28 - September 5, 2015, Erice, Italy, August 31, 2015.

• R. Henrion, Application of chance constraints in a coupled model of hydro-wind energy production, Charles University in Prague, Faculty of Mathematics and Physics, Czech Republic, March 6, 2014.

• R. Henrion, Application of probabilistic constraints to problems of energy management under uncertainty, Eidgenössische Technische Hochschule Zürich, Power Systems Laboratory, Switzerland, September 30, 2014.

• R. Henrion, Conditioning of linear-quadratic two-stage stochastic optimization problems, 5th Conference on Optimization Theory and its Applications (ALEL 2014), June 5 - 7, 2014, Universidad de Sevilla, Spain, June 6, 2014.

• R. Henrion, Gradient formulae in probabilistic programming, Conference on Optimization & Practices in Industry (PGMO-COPI'14), October 28 - 31, 2014, Ecole Polytechnique, Paris, France, October 29, 2014.

• R. Henrion, Gradient formulae in probabilistic programming, Université Paul Sabatier, Laboratoire d'analyse et d'architecture des systèmes, Toulouse, France, December 8, 2014.

• R. Henrion, Nonlinear programming for solving chance constrained optimization problems: Application to renewable energies, Winter School on Stochastic Programming with Applications in Energy, Finance and Insurance, March 23 - 28, 2014, Bad Hofgastein, Austria, March 25, 2014.

• R. Henrion, Probabilistic constraints in hydro reservoir management, XIII Symposium of Specialists in Electric Operational and Expansion Planning (SEPOPE), May 18 - 21, 2014, Foz do Iguassu, Brazil, May 19, 2014.

• R. Henrion, Probabilistic constraints via nonlinear programming: Application to energy management problems, Euro Mini Conference on Stochastic Programming and Energy Applications (EuroCSP2014), September 24 - 26, 2014, Institut Henri Poincaré, Paris, France, September 25, 2014.

• R. Henrion, Probabilistic constraints: A structure-oriented introduction, Optimization and Applications Seminar, Eidgenössische Technische Hochschule Zürich, Switzerland, September 29, 2014.

• R. Henrion, Problèmes d'optimisation sous contraintes en probabilité: une initiation, December 9 - 10, 2014, Université Paul Sabatier, Institut de Mathématiques de Toulouse, France.

### Preprints im Fremdverlag

• F. Stonyakin, D. Dvinskikh, P. Dvurechensky, A. Kroshnin, O. Kuznetsova, A. Agafonov, A. Gasnikov, A. Tyurin, C.A. Uribe, D. Pasechnyuk, S. Artamonov, Gradient methods for problems with inexact model of the objective, Preprint no. arXiv:1902.09001, Cornell University Library, arXiv.org, 2019.

• D. Dvinskikh, E. Gorbunov, A. Gasnikov, P. Dvurechensky, C.A. Uribe, On dual approach for distributed stochastic convex optimization over networks, Preprint no. arXiv:1903.09844, Cornell University Library, arXiv.org, 2019.
Abstract
We introduce dual stochastic gradient oracle methods for distributed stochastic convex optimization problems over networks. We estimate the complexity of the proposed method in terms of probability of large deviations. This analysis is based on a new technique that allows to bound the distance between the iteration sequence and the solution point. By the proper choice of batch size, we can guarantee that this distance equals (up to a constant) to the distance between the starting point and the solution.

• P. Dvurechensky, A. Gasnikov, E. Gorbunov, An accelerated directional derivative method for smooth stochastic convex optimization, Preprint no. arXiv:1804.02394, Cornell University Library, arXiv.org, 2018.
Abstract
We consider smooth stochastic convex optimization problems in the context of algorithms which are based on directional derivatives of the objective function. This context can be considered as an intermediate one between derivative-free optimization and gradient-based optimization. We assume that at any given point and for any given direction, a stochastic approximation for the directional derivative of the objective function at this point and in this direction is available with some additive noise. The noise is assumed to be of an unknown nature, but bounded in the absolute value. We underline that we consider directional derivatives in any direction, as opposed to coordinate descent methods which use only derivatives in coordinate directions. For this setting, we propose a non-accelerated and an accelerated directional derivative method and provide their complexity bounds. Despite that our algorithms do not use gradient information, our non-accelerated algorithm has a complexity bound which is, up to a factor logarithmic in problem dimension, similar to the complexity bound of gradient-based algorithms. Our accelerated algorithm has a complexity bound which coincides with the complexity bound of the accelerated gradient-based algorithm up to a factor of square root of the problem dimension, whereas for existing directional derivative methods this factor is of the order of problem dimension. We also extend these results to strongly convex problems. Finally, we consider derivative-free optimization as a particular case of directional derivative optimization with noise in the directional derivative and obtain complexity bounds for non-accelerated and accelerated derivative-free methods. Complexity bounds for these algorithms inherit the gain in the dimension dependent factors from our directional derivative methods.

• P. Dvurechensky, A. Gasnikov, E. Gorbunov, An accelerated method for derivative-free smooth stochastic convex optimization, Preprint no. arXiv:1802.09022, Cornell University Library, arXiv.org, 2018.
Abstract
We consider an unconstrained problem of minimization of a smooth convex function which is only available through noisy observations of its values, the noise consisting of two parts. Similar to stochastic optimization problems, the first part is of a stochastic nature. On the opposite, the second part is an additive noise of an unknown nature, but bounded in the absolute value. In the two-point feedback setting, i.e. when pairs of function values are available, we propose an accelerated derivative-free algorithm together with its complexity analysis. The complexity bound of our derivative-free algorithm is only by a factor of n??? larger than the bound for accelerated gradient-based algorithms, where n is the dimension of the decision variable. We also propose a non-accelerated derivative-free algorithm with a complexity bound similar to the stochastic-gradient-based algorithm, that is, our bound does not have any dimension-dependent factor. Interestingly, if the solution of the problem is sparse, for both our algorithms, we obtain better complexity bound if the algorithm uses a 1-norm proximal setup, rather than the Euclidean proximal setup, which is a standard choice for unconstrained problems.

• P. Dvurechensky, D. Dvinskikh, A. Gasnikov, C.A. Uribe, A. Nedić, Decentralize and randomize: Faster algorithm for Wasserstein barycenters, Preprint no. arXiv:1806.03915, Cornell University Library, arXiv.org, 2018.
Abstract
We study the problem of decentralized distributed computation of a discrete approximation for regularized Wasserstein barycenter of a finite set of continuous probability measures distributedly stored over a network. Particularly, we assume that there is a network of agents/machines/computers where each agent holds a private continuous probability measure, and seeks to compute the barycenter of all the measures in the network by getting samples from its local measure and exchanging information with its neighbors. Motivated by this problem, we develop and theoretically analyze a novel accelerated primal-dual stochastic gradient method for general stochastic convex optimization problems with linear equality constraints. Then, we apply this method to the decentralized distributed optimization setting to propose a new algorithm for the distributed semi-discrete regularized Wasserstein barycenter problem. The proposed algorithm can be executed over arbitrary networks that are undirected, connected and static, using the local information only. Moreover, we show explicit non-asymptotic complexity in terms of the problem parameters. Finally, we show the effectiveness of our method on the distributed computation of the regularized Wasserstein barycenter of univariate Gaussian and von Mises distributions, as well as on some applications to image aggregation. ## Ansprechpartner ## Beteiligte Gruppen des WIAS ## Anwendungen ## Projekte/Drittmittel 