Elementary Mathematical Methods - Session 1

Those people who want to register for the course, send me an email at george@madscitech.org. You may connect up with us tonight at 8 PM Central Time on Skype. Use my Skype name georgehrab. If you are not registered for the course, I will not respond.

The Book is Out!!!

Well, the book I wrote with Leonard Susskind, The Theoretical Minimum, is out and is a huge success! Thanks to all of you who have been following this! You can find the web site to support the book at www.madscitech.org/tm.

New Information Theory Post and Mathematica 9

Mathematica 9 has come out and has some amazing new capabilities. I have produced a new information theory post using some of the new functions. This is at http://www.madscitech.org/information/information.html

The Web Page for The Theoretical Minimum is Up!

The web page for The Theoretical Minimum is up and you can see it at: www.madscitech.org/tm.

Book Update

The book, The Theoretical Minimum, will be published by Basic Books at the end of January of 2013. There will be a companion web site: www.madscitech.org/tm, it is not yet up! Don't go there and then tell me there is no link!

New Writing on Information Theory: Some important theorems on Markov Chains, their proofs, and how to model Markov Chains in Mathematica. http://www.madscitech.org/information/information.html

New Information Theory Site on MAST

I have formalized my notes on information theory and placed them on the MAST website. You can find tham at http://www.madscitech.org/information/information.html

More Information on the Book

I just finished working over the proofs for the publsher. It is now in their hands again. The next step is the Galleys, and then back to the publisher again. The book is scheduled to be out in January 2012.

The Nature of Information V

In this installment I demonstrate how to determine some of the quantities we have been looking into using Mathematica 8. You can find it at this website:

http://www.madscitech.org/csg/it.html

under Information Theory V.

The Nature of Information IV

In this post I examine what coding is and introduce Shannon's First Coding Theorem and the Shannon-McMillan Theorem. See Information Theory IV.

The Nature of information III

In this posting I explore the probability of finding strings and establish the principle that the chace of finding a given string is independent from the elements of the string. This result is called the Asymptotic Equipartition Property. For details go to the The Nature of Information III

The Nature of Information II

The Nature of Information II

George Hrabovsky

MAST

Introduction

The previous article was concerned primary with a sequences of events that were mutually exclusive. What happens when more than one such sequence, or scheme, are combined?

Combining Schemes—Mutually Independent Sets

Let’s say that we have two finite schemes,

and

Say that the propbability of the joint occurence of events

 and

 

is

This property is called a mutually independent probability. In fact the set of events

forms another finite scheme. The entropy for such a scheme is,

Combining Schemes—Mutually Dependent Sets

So what if A and B are not mutually independent? Here we say that the probability of event

 
of the scheme B occurs assuming that event

 
occurs from scheme A is

This gives us the scheme

with entropy

It turns out that

 
is a random variable called the expectation of H(B) in the scheme A. It is also written

.

What does It all Mean?

The amount of information delivered by a scheme never decreases unless a another scheme is realized beforehand.

The Nature of information

The Nature of Information

George Hrabovsky

MAST

Introduction

Information can be thought of as what we do not already know. We acquire information when we learn, or perhaps it is that the act of learning is acquiring information. In either case we can consider infomration in the abstract to be a phsycial quantity and it can be measured.

Bits

How do we measure information? Like any measurement of a physical quantity we choose a fundamental unit. Then we say that something has a quantity of units, a numerical result. Something can be five meters long, or it can be 4.3 pounds of weight, or 6.1 kilograms of mass, and so on. For information we use the bit as a fundamental quantity. Thus we can say that some information is composed of a number of bits.

Characterizing Information

We will use symbols to represent bits of information. A set of symbols comprising a single bit will be represented by

two bits by

 

and so on, so that a set of n such symbols is the sequence

.

Schemes

If we also associate a probability that a given symbol will be sent,

The symbols of such a set must be mutually exclusive, so that the total of all probabilies is 1. We can combine them into a single symbol,

This is called a finite scheme.

Entropy

Entropy is, among other things, the measure of the disorder of a system of symbols. In the case of a scheme the entropy is given by,

We will discuss these things in more detail later.

A New Inquiry—Information Theory

Diving into Information Theory

I have begun to delve into the mysteries of information theory. I plan to post the results here. I cannot predict where it will lead, but it will be great fun. For now I plan the following:

1) The nature of information.
2) Channel capacity.
3) Gaussian channels.
4) Fisher information.
5) The thermodynamics of information.
6) The statistical mechanics of information.
7) Ergodic sources.
8) Entropy
9) Detecting information.
10) Measure of information.
11) Encoding.
12) Noiseless coding.
13) Discrete channels.
14) Error-correcting codes.
15) Channels with memory.
16) Continuous channels.

It is possible that I will get to quantum information and black holes.

Book Deal

Today I received the executed book deal for my collaboration with Leonard Susskind on the book Theoretical Minimum: Classical Mechanics. The book will be published by Basic Books. We have until January of 2012 to finish it, but don't be surprised if we finish early.

Book News

Leonard Susskind and I have now signed a contract with Basic Books for the Theoretical Minimum: Classical Mechanics. This book is contracted to be completed by January of 2012 (don't be surprised if it is finished before then).

The Book Is DONE!!!

At 05:17 on Monday, 31 January 2011 I finished the first draft of the book Theoretical Minimum I: Classical Mechanics by Leonard Susskind and George Hrabovsky. This book was written in Mathematica 7 and Mathematica 8.

The Book is Almost Done!

I am almost done with the final two cahpters of the book I am writing with Leonard Susskind! I will be done by Sunday!

Book Project Update

Well here I am again. Sorry about the lengthy delay in posting, I am in a full-scale assault on the book to finish it by the end of this week. I have completed chapter 16, am finishing up chapter 15, and will finish 17 and 18 by week's end! The first draft is nearly done!

Here is the book now:

Lecture 1: Dynamical systems.

Interlude 1: Calculus.

Interlude 2: Newtonian mechanics.

Lecture 2: Conservation laws, the principle of least action, and the Lagrangian.

Interlude 3: Vectors and vector spaces.

Lecture 3: Conservation laws imply symmetry.

Interlude 4: Symmetry.

Lecture 4: Time translation symmetry, the conservation of energy, and the Hamiltonian.

Interlude 5: Transformations and matrices.

Lecture 5: Pendulums and springs.

Interlude 6: Complex numbers and ordinary differential equations. 

Lecture 6: The Hamiltonian method of mechanics. 

Interlude 7: Vector analysis.

Lecture 7: Liouville's Theorem and Information Conservation

Interlude 8: Thermodynamics (almost complete)

Lecture 8: Charged Particles in Electromagnetic Fields and an Introduction to Poisson Brackets

Interlude 9: Chaos (not done yet)

Lecture 9: Poisson Brackets and Canonical Transformations

Some new results

MAST has begun to post some research results!

1) I have started to do some dripping handrail models of accretion disks. You can find them here: http://www.madscitech.org/csg/accretion.html

2) James Firmiss and I have begun serious work on neutron star equations of state, results will be put up shortly.

3) Chris Winfield is giving a talk on local solvability at the American Mathematical Society meeting in New Orleans, that talk is located here: http://www.madscitech.org/csg/AMS10mintalk.pdf

4) We are doing research and development of database capabilities in Mathematica, mostly due to the efforts of Rocky Wenz.

More results will be reported as they occur!