We are open Saturday and Sunday!

Call Now to Set Up Tutoring:

(888) 888-0446

Math Modeling : Markov Chains & Processes

Study concepts, example questions & explanations for Math Modeling

Create An Account

All Math Modeling Resources

20 Practice Tests Question of the Day Flashcards Learn by Concept

Example Questions

Example Question #1 : Markov Chains & Processes

A computer company has one service repair man and has space for 29 computers in the shop at one time. Last year the shop repaired 67 computers with an average repair time of 2 days per computer. Formulate a Markov process model for X_t which represents the number of computers in the shop for repair at time months and calculate the EX_t .

Possible Answers:

$EX_t\approx 1.067$

$EX_t\approx 1.307$

$EX_t\approx 1.133$

$EX_t\approx 1.036$

$EX_t\approx 1.037$

Correct answer:

$EX_t\approx 1.037$

Explanation:

Using Markov Process for this question assume that the space is finite.

$X_t\epsilon\begin{Bmatrix} 1,2,3,4,...,m \end{Bmatrix}$

The stochastic process which determines the future of the process from a probabilistic standpoint is defined as follows.

Let

$T_i=\text{time spent in state }i$

$Pr{\begin{Bmatrix} T_i>t+s|T_i>s \end{Bmatrix}}=Pr\begin{Bmatrix} T_i>t \end{Bmatrix}$

Recall that the exponential distribution, and thus the T_i could have the density function,

$F_i(t)=\lambda_ie^{-\lambda_it}$

Now formulate the Markov Process specifically for this problem.

$X_t\epsilon\begin{Bmatrix} 0,1,2,..., 29\end{Bmatrix}$

The transitions of computers in and out of the shop are,

X_t=i to X_t=i+1 or X_t=i-1

Now calculate the rates up and down.

To calculate $\lambda$ take the total number of computers fixed last year and divide it by the total months in the year. To calculate $\mu$ assume there are 22 working days in a month and it takes on average, two days to fix a computer.

$\\\lambda=5.6 \\\mu=11$

Its key to know that at zero we cannot move down a state and like wise when at 29, we cannot move up a state.

$[\text{Rate In}]=[\text{Rate Out}]$

$\\\lambda P_0=\mu P_1 \\(\mu +\lambda)P_1=\lambda P_0+\mu P_1 \\(\mu +\lambda)P_2=\lambda P_2+\mu P_3\\ \begin{matrix} .\\ .\\ . \end{matrix} \\\mu P_{29}=\lambda P_{28}$

Solving with

$\sum P_i=1$

$EX_t=\sum iP_i$

Continuing in this fashion results in the following.

$\\P_n=(\lambda / \mu)P_{n-1} \\P_n=(\lambda / \mu)^nP_0 \\\\\sum_{n=0}^{29}P_n=P_0\sum_{n=0}^{29}\left(\frac{\lambda}{\mu} \right )^n=1 \\\\P_0=\frac{1-\rho }{1-\rho^{30}}$

Now, writing the sum of a finite geometric series is

$P_n=\rho^nP_0=\frac{\rho^n(1-\rho)}{1-\rho^{30}}$

Now

$\rho=\frac{\lambda}{\mu}=\frac{5.6}{11}=0.509$

therefore,

$1-\rho^{30}=1-0.509^{30}\approx 0.9999999984$

Next,

$Pr\begin{Bmatrix} X_t>0 \end{Bmatrix}=1-P_0=\rho\approx 0.509$

$\\EX_t=\sum_{n=0}^{29}nP_n \\EX_t=\sum_{n=0}^{29}n\rho^n(1-\rho) \\EX_t\approx 1.037$

Report an Error

Example Question #1 : Probability Models

A computer company has one service repair man and has space for 32 computers in the shop at one time. Last year the shop repaired 71 computers with an average repair time of 3 days per computer. Formulate a Markov process model for X_t which represents the number of computers in the shop for repair at time months and calculate the EX_t .

Possible Answers:

$EX_t\approx 4.761$

$EX_t\approx 4.236$

$EX_t\approx 4.716$

$EX_t\approx 4.426$

$EX_t\approx 4.176$

Correct answer:

$EX_t\approx 4.176$

Explanation:

Using Markov Process for this question assume that the space is finite.

$X_t\epsilon\begin{Bmatrix} 1,2,3,4,...,m \end{Bmatrix}$

The stochastic process which determines the future of the process from a probabilistic standpoint is defined as follows.

Let

$T_i=\text{time spent in state }i$

$Pr{\begin{Bmatrix} T_i>t+s|T_i>s \end{Bmatrix}}=Pr\begin{Bmatrix} T_i>t \end{Bmatrix}$

Recall that the exponential distribution, and thus the T_i could have the density function,

$F_i(t)=\lambda_ie^{-\lambda_it}$

Now formulate the Markov Process specifically for this problem.

$X_t\epsilon\begin{Bmatrix} 0,1,2,..., 32\end{Bmatrix}$

The transitions of computers in and out of the shop are,

X_t=i to X_t=i+1 or X_t=i-1

Now calculate the rates up and down.

$\\\lambda=5.9 \\\mu=7.3$

Its key to know that at zero we cannot move down a state and like wise when at 29, we cannot move up a state.

$[\text{Rate In}]=[\text{Rate Out}]$

$\\\lambda P_0=\mu P_1 \\(\mu +\lambda)P_1=\lambda P_0+\mu P_1 \\(\mu +\lambda)P_2=\lambda P_2+\mu P_3\\ \begin{matrix} .\\ .\\ . \end{matrix} \\\mu P_{32}=\lambda P_{31}$

Solving with

$\sum P_i=1$

$EX_t=\sum iP_i$

Continuing in this fashion results in the following.

$\\P_n=(\lambda / \mu)P_{n-1} \\P_n=(\lambda / \mu)^nP_0 \\\\\sum_{n=0}^{32}P_n=P_0\sum_{n=0}^{32}\left(\frac{\lambda}{\mu} \right )^n=1 \\\\P_0=\frac{1-\rho }{1-\rho^{33}}$

Now, writing the sum of a finite geometric series is

$P_n=\rho^nP_0=\frac{\rho^n(1-\rho)}{1-\rho^{30}}$

Now

$\rho=\frac{\lambda}{\mu}=\frac{5.9}{7.3}=0.808$

therefore,

$1-\rho^{33}=1-0.808^{33}\approx 0.9991198131$

Next,

$Pr\begin{Bmatrix} X_t>0 \end{Bmatrix}=1-P_0=\rho\approx 0.808$

$\\EX_t=\sum_{n=0}^{32}nP_n \\EX_t=\sum_{n=0}^{32}n\rho^n(1-\rho) \\EX_t\approx 4.176$

Report an Error

Example Question #3 : Markov Chains & Processes

A computer company has one service repair man and has space for 23 computers in the shop at one time. Last year the shop repaired 51 computers with an average repair time of 5 days per computer. Formulate a Markov process model for X_t which represents the number of computers in the shop for repair at time months and calculate the EX_t .

Possible Answers:

$EX_t\approx 0.997$

$EX_t\approx 1.999$

$EX_t\approx 0.989$

$EX_t\approx 0.999$

$EX_t\approx 1.989$

Correct answer:

$EX_t\approx 0.999$

Explanation:

Using Markov Process for this question assume that the space is finite.

$X_t\epsilon\begin{Bmatrix} 1,2,3,4,...,m \end{Bmatrix}$

The stochastic process which determines the future of the process from a probabilistic standpoint is defined as follows.

Let

$T_i=\text{time spent in state }i$

$Pr{\begin{Bmatrix} T_i>t+s|T_i>s \end{Bmatrix}}=Pr\begin{Bmatrix} T_i>t \end{Bmatrix}$

Recall that the exponential distribution, and thus the T_i could have the density function,

$F_i(t)=\lambda_ie^{-\lambda_it}$

Now formulate the Markov Process specifically for this problem.

$X_t\epsilon\begin{Bmatrix} 0,1,2,..., 23\end{Bmatrix}$

The transitions of computers in and out of the shop are,

X_t=i to X_t=i+1 or X_t=i-1

Now calculate the rates up and down.

$\\\lambda=2.2 \\\mu=4.4$

Its key to know that at zero we cannot move down a state and like wise when at 29, we cannot move up a state.

$[\text{Rate In}]=[\text{Rate Out}]$

$\\\lambda P_0=\mu P_1 \\(\mu +\lambda)P_1=\lambda P_0+\mu P_1 \\(\mu +\lambda)P_2=\lambda P_2+\mu P_3\\ \begin{matrix} .\\ .\\ . \end{matrix} \\\mu P_{23}=\lambda P_{22}$

Solving with

$\sum P_i=1$

$EX_t=\sum iP_i$

Continuing in this fashion results in the following.

$\\P_n=(\lambda / \mu)P_{n-1} \\P_n=(\lambda / \mu)^nP_0 \\\\\sum_{n=0}^{23}P_n=P_0\sum_{n=0}^{23}\left(\frac{\lambda}{\mu} \right )^n=1 \\\\P_0=\frac{1-\rho }{1-\rho^{24}}$

Now, writing the sum of a finite geometric series is

$P_n=\rho^nP_0=\frac{\rho^n(1-\rho)}{1-\rho^{30}}$

Now

$\rho=\frac{\lambda}{\mu}=\frac{2.2}{4.4}=0.5$

therefore,

$1-\rho^{24}=1-0.5^{24}\approx 0.9999999404$

Next,

$Pr\begin{Bmatrix} X_t>0 \end{Bmatrix}=1-P_0=\rho\approx 0.5$

$\\EX_t=\sum_{n=0}^{23}nP_n \\EX_t=\sum_{n=0}^{23}n\rho^n(1-\rho) \\EX_t\approx 0.999$

Report an Error

View Tutors

Andrew
Certified Tutor

University of Waterloo, Bachelor, Math.

View Tutors

Ian
Certified Tutor

Seneca College, Associate, Recreation Facility Management. Texas A&M University - College Station, Bachelor, Recreation & Spo...

View Tutors

Jyotsna
Certified Tutor

University of Strathclyde, Master's/Graduate, Advanced Immunology.

All Math Modeling Resources

20 Practice Tests Question of the Day Flashcards Learn by Concept

Popular Subjects

GMAT Tutors in Chicago, LSAT Tutors in San Diego, Math Tutors in Atlanta, Math Tutors in New York City, Algebra Tutors in New York City, SAT Tutors in Atlanta, Spanish Tutors in Washington DC, GRE Tutors in San Diego, MCAT Tutors in Denver, French Tutors in San Diego

Popular Courses & Classes

LSAT Courses & Classes in Miami, LSAT Courses & Classes in Phoenix, ACT Courses & Classes in Houston, ACT Courses & Classes in New York City, GRE Courses & Classes in Seattle, SAT Courses & Classes in Miami, Spanish Courses & Classes in Chicago, ISEE Courses & Classes in New York City, SSAT Courses & Classes in Seattle, GRE Courses & Classes in Atlanta

Popular Test Prep

SAT Test Prep in Houston, GRE Test Prep in Los Angeles, LSAT Test Prep in Miami, SAT Test Prep in Atlanta, ISEE Test Prep in Atlanta, LSAT Test Prep in Dallas Fort Worth, GMAT Test Prep in Phoenix, LSAT Test Prep in Phoenix, SAT Test Prep in Miami, MCAT Test Prep in Atlanta

DMCA Complaint

If you believe that content available by means of the Website (as defined in our Terms of Service) infringes one or more of your copyrights, please notify us by providing a written notice (“Infringement Notice”) containing the information described below to the designated agent listed below. If Varsity Tutors takes action in response to an Infringement Notice, it will make a good faith attempt to contact the party that made such content available by means of the most recent email address, if any, provided by such party to Varsity Tutors.

Your Infringement Notice may be forwarded to the party that made the content available or to third parties such as ChillingEffects.org.

Please be advised that you will be liable for damages (including costs and attorneys’ fees) if you materially misrepresent that a product or activity is infringing your copyrights. Thus, if you are not sure content located on or linked-to by the Website infringes your copyright, you should consider first contacting an attorney.

Please follow these steps to file a notice:

You must include the following:

A physical or electronic signature of the copyright owner or a person authorized to act on their behalf; An identification of the copyright claimed to have been infringed; A description of the nature and exact location of the content that you claim to infringe your copyright, in \ sufficient detail to permit Varsity Tutors to find and positively identify that content; for example we require a link to the specific question (not just the name of the question) that contains the content and a description of which specific portion of the question – an image, a link, the text, etc – your complaint refers to; Your name, address, telephone number and email address; and A statement by you: (a) that you believe in good faith that the use of the content that you claim to infringe your copyright is not authorized by law, or by the copyright owner or such owner’s agent; (b) that all of the information contained in your Infringement Notice is accurate, and (c) under penalty of perjury, that you are either the copyright owner or a person authorized to act on their behalf.

Send your complaint to our designated agent at:

Charles Cohn Varsity Tutors LLC
101 S. Hanley Rd, Suite 300
St. Louis, MO 63105

Or fill out the form below:

Do not fill in this field

Contact Information

* Full legal name

Company name

* Phone number

* Email address

Address

* Address1

Address2

* City

* State

* Zipcode

* Country

Complaint Details

* URL of copyrighted work (where your original material is located)

* Please describe the copyrighted work so that it may be easily identified

I am the owner, or an agent authorized to act on behalf of the owner of an exclusive right that is allegedly infringed.

I have a good faith belief that the use of the material in the manner complained of is not authorized by the copyright owner, its agent, or the law.

This notification is accurate.

I acknowledge that there may be adverse legal consequences for making false or bad faith allegations of copyright infringement by using this process.

* Signature (your digital signature is legally binding)

HIGH SCHOOL

GRADUATE SCHOOL

K-8

Search 50+ Tests

math tutoring

science tutoring

foreign languages

elementary tutoring

other

Search 350+ Subjects

Math Modeling : Markov Chains & Processes

Study concepts, example questions & explanations for Math Modeling

All Math Modeling Resources

Example Questions

Example Question #1 : Markov Chains & Processes

Example Question #1 : Probability Models

Example Question #3 : Markov Chains & Processes

All Math Modeling Resources

Report an issue with this question

DMCA Complaint

Contact Information

Address

Complaint Details

Email address:
Your name:
Feedback: