Optimization Days 2017

HEC Montréal, May 8-10, 2017

1st Canadian Healthcare Optimization Workshop (CHOW)

HEC Montréal, May 10-11, 2017



HEC Montréal, 8 — 11 May 2017

Schedule Authors My Schedule

WB7 Processus de Markov / Markov decision processes

May 10, 2017 10:30 AM – 12:10 PM

Location: St-Hubert

3 Presentations

  • 10:30 AM - 10:55 AM

    Renewal theory based reinforcement learning for Markov processes with controlled restarts

    • Jayakumar Subramanian, presenter, McGill
    • Jhelum Chakravorty,
    • Aditya Mahajan, GERAD, McGill University

    Markov processes with controlled restarts arise in networked control systems.
    Under a threshold based strategy, such processes are regenerative. Therefore,
    the optimal performance can be written in terms of the performance
    during a regenerative cycle. We exploit this relationship to develop a
    sample-path based policy gradient algorithm.

  • 10:55 AM - 11:20 AM

    ANNULE / A Markov-modulated End-to-end Delay Analysis of Large-scale RF-Mesh Networks with Time-slotted ALOHA and FHSS

    • Filippo Malandra, presenter, Politecnico di Milano
    • Brunilde Sansò, Ecole Polytechnique de Montreal

    A new mathematical model and a methodology are proposed to evaluate the performance of large scale
    RF-Mesh Networks that use time-slotted ALOHA with Frequency Hopping Spread Spectrum. An
    analytic formulation for the delay, based on Markov-modulated modelling of the system, is derived. The
    formula can be extended to evaluate other important performance metrics. The proposed methodology
    is applied to a large scale network of several thousands of nodes, and numerical results are reported to
    show the wide variety of performance evaluations that are enabled. The usefulness of the assessment of
    the feasibility of different types of applications (e.g., smart-metering, sensor networks, IoT) is shown.
    An analysis of the scalability of this methodology and a comparison with simulation results are also

  • 11:20 AM - 11:45 AM

    Global Inventory Planning with Loosely Coupled Markov Decision processes

    • Thierry Moisan, presenter, Université Laval
    • Éric Prescott Gagnon, JDA Software
    • Yossiri Adulyasak, HEC Montréal

    We present a general approach to plan the inventory level of slow-moving items where service level targets are applied on a set of items. Loosely coupled Markov decision processes are used within a column generation algorithm with the objective of minimizing overall costs while satisfying service level targets.