Friday, May 18, 2012

The Nature of information

The Nature of Information

George Hrabovsky
MAST

Introduction

Information can be thought of as what we do not already know. We acquire information when we learn, or perhaps it is that the act of learning is acquiring information. In either case we can consider infomration in the abstract to be a phsycial quantity and it can be measured.

Bits

How do we measure information? Like any measurement of a physical quantity we choose a fundamental unit. Then we say that something has a quantity of units, a numerical result. Something can be five meters long, or it can be 4.3 pounds of weight, or 6.1 kilograms of mass, and so on. For information we use the bit as a fundamental quantity. Thus we can say that some information is composed of a number of bits.

Characterizing Information

We will use symbols to represent bits of information. A set of symbols comprising a single bit will be represented by
two bits by
 
and so on, so that a set of n such symbols is the sequence

.

Schemes

If we also associate a probability that a given symbol will be sent,

The symbols of such a set must be mutually exclusive, so that the total of all probabilies is 1. We can combine them into a single symbol,
This is called a finite scheme.

Entropy

Entropy is, among other things, the measure of the disorder of a system of symbols. In the case of a scheme the entropy is given by,
We will discuss these things in more detail later.

Thursday, May 17, 2012

A New Inquiry—Information Theory

Diving into Information Theory

I have begun to delve into the mysteries of information theory. I plan to post the results here. I cannot predict where it will lead, but it will be great fun. For now I plan the following:

1) The nature of information.
2) Channel capacity.
3) Gaussian channels.
4) Fisher information.
5) The thermodynamics of information.
6) The statistical mechanics of information.
7) Ergodic sources.
8) Entropy
9) Detecting information.
10) Measure of information.
11) Encoding.
12) Noiseless coding.
13) Discrete channels.
14) Error-correcting codes.
15) Channels with memory.
16) Continuous channels.

It is possible that I will get to quantum information and black holes.

Wednesday, March 14, 2012

New paper published.

I just had a paper published in The Mathematica Journal. You can find it here.

Sunday, March 4, 2012

On the nature of theoretical physics.

If you are reading this, you are interested in theoretical physics. But what is theoretical physics? I have come up with several definitions, based on how you approach the subject.

1) The modeling approach to theoretical physics. Another way of calling this would be the phenomena-centered approach, whose goal is to understand a specific phenomena by developing either a mathematical or computational model. You begin this by choosing a phenomena to study. Then you choose an approach to representing the phenomena; can you represent it as particle? a field? or some continuous distribution of matter? Then you choose a mathematical formulation. Examples of mathematical formulations are Newtonian mechanics, Maxwell's equations, Lorentz covariance, the Maxwell-Boltzmann distribution, etc. Such formulations Constitute much of the material of most textbooks and courses on physics. You then adapt your approach to the mathematical formulation, thus developing a mathematical representation of your phenomena. You then use physical, mathematical, and/or computational arguments and methods to make predictions in the form of tables, plots, and/or formulas. By studying these results in different circumstances you can extend our understanding of the phenomena. This is the most direct method of doing theoretical physics, it is a straight application of mathematical or computational methods. It is certainly the most structured way of doing theoretical physics.

2) The constructive approach to theoretical physics. This can be thought of as the method to develop a new formulation of a physical theory. Examples are the Lagrangian formulation of mechanics, the Lagrangian formulation of electrodynamics, the Eulerian formulation of fluid dynamics, the path-integral formulation of quantum mechanics, and so on. You begin by choosing how you represent objects in your developing theory. Then you choose some quantity, or set of quantities to base your construction on. Then you choose an argument to base your construction on.  Are you seeking to find symmetries? Are you arguing from some conserved quantity? Are you assuming that your quantity is minimized? For example, in the Lagrangian formulation you choose to create a new quantity called the Lagrangian and then you work out the consequences when the integral of the Lagrangian—the action—is minimized. This leads to the Euler-Lagrange equations of motion, an new formulation of classical mechanics. This is a much more difficult, but powerful method—you build the formulation. The difficulty stems from the lack of structural guidelines in creating a new formulation.

3) The abstract approach to theoretical physics. This mode is where you take a number of specific cases and generalize their results. For example, knowing that when a derivative is 0 and quantity is unchanged; you take the zero derivatives of momentum in many cases and generalize that into the law of conservation of momentum. This sort of activity is very difficult since there are few guidelines for how to proceed beyond what is already known.

4) The unification approach to theoretical physics. This is based on the idea that it would be nice if there was a single theory to govern a wide range of phenomena. There is no real reason to believe that this is true generally. This is one difficulty with practical application. another difficulty is that all of our equations are, to one degree or another, an approximation of reality. So the fact that equations in different fields look alike is another way of saying that the approximations are similar. Does that mean the phenomena are also similar? Sometimes. Isaac Newton unified gravity at the surface of the Earth and gravity away from the Earth. James Maxwell unified electricity, magnetism, and light. Abdus Salam, Sheldon Glashow and Steven Weinberg unified electromagnetism and the weak nuclear force. The work of unifying electroweak theory with the strong interaction force is a work in progress. Even less success has been made in unifying gravity.

So this, then, is the general nature of theoretical physics.