Accelerating the Solution of Linear Systems by Iterative Refinement in Three Precisions

by Erin Carson and Nick Higham

lu-ir-hsd.jpg

Half precision LU factorization (H,S D) can deliver full single precision accuracy (Algorithm New), just like traditional iterative refinement (S,S,D: Algorithm Trad).

With the growing availability of half precision arithmetic in hardware and quadruple precision arithmetic in software, it is natural to ask whether we can harness these different precisions, along with the standard single and double precisions, to solve problems faster or more accurately.

We have been investigating this question for linear systems Ax = b with a nonsingular matrix A, for which the standard solution process is by LU factorization. By making use of iterative refinement, we are able to harness an LU factorization computed in lower precision to solve the problem up to twice as fast and with greater accuracy than with the standard approach.

Iterative refinement is an old technique for improving the accuracy of an approximate solution to a linear system Ax = b. It was used by Wilkinson and his colleagues in the 1940s in the early days of digital computing. The traditional usage employs two precisions, and fixed precision refinement has also been in use since the late 1970s.

We have found that using three different precisions, rather than the usual two, can bring major benefits. A scenario of particular interest is a mix of half precision, single precision, and double precision, with single precision the working precision in which A, b, and the iterates x_i are stored. Here is the traditional algorithm followed by the new algorithm. All computations are in single precision (unit roundoff 6.0 \times 10^{-8}) except where stated.

Algorithm Trad: two-precision refinement (single, double).
Factorize $PA = LU$.
Solve $Ax_0 = b$ using the LU factors.
for $i=0:\infty$
   Form $r_i = b - Ax_i$ in *double precision*
        and round $r_i$ to *single precision*.
   Solve $Ad_i = r_i$ using the LU factors.
   $x_{i+1} = x_i + d_i$.
end
Algorithm New: three-precision refinement (half, single, double).
Factorize $PA = LU$ in *half precision*.
Solve $Ax_0 = b$ using the LU factors at *half precision*.
for $i=0:\infty$
   Form $r_i = b - Ax_i$ in *double precision*
        and round $r_i$ to *half precision*.
   Solve $Ad_i = r_i$ at *half precision*
        using the LU factors; store $d_i$ in *single*.
 $x_{i+1} = x_i + d_i$.
end

Speed

Algorithm Trad does O(n^3) flops at single precision and O(n^2) flops at double precision. Algorithm New, however, does O(n^3) flops at half precision and O(n^2) flops at single and double precision. Both these statements assume, of course, that iterative refinement converges in a small number of iterations. There is therefore a potential two times speedup of Algorithm New over Algorithm Trad, since half precision runs at twice the speed of single precision on (for example) NVIDIA GPUs and AMD GPUs.

Accuracy

Algorithm Trad converges as long as \kappa_{\infty}(A) \le 10^8 and it yields a forward error (defined by \|x-\widehat{x}\|_{\infty}/\|x\|_{\infty}, where \widehat{x} is the computed solution) and a backward error both of order 10^{-8} (as shown by standard analysis). Our new rounding error analysis shows that Algorithm New has the same error bounds, but has the more stringent requirement \kappa_{\infty}(A) \le 10^4 for convergence.

GMRES-IR

Now we replace the solve step in the loop of Algorithm New by an application of GMRES to the preconditioned system

\widetilde{A} d_i \equiv \widehat{U}^{-1}\widehat{L}^{-1}Ad_i = \widehat{U}^{-1}\widehat{L}^{-1}r_i,

where matrix–vector products with \widetilde{A} are done at double precision and all other computations are done at single precision. Algorithm New now converges as long as \kappa_{\infty}(A) \le 10^8 and it yields forward and backward errors of order 10^{-8}. In other words, it has the same numerical properties as Algorithm Trad but potentially does half the work (depending on the number of GMRES iterations needed to converge).

Other Choices of Precision

Let H, S, D, and Q denote half precision, single precision, double precision, and quadruple precision, respectively. Algorithm New can be described as “HSD”, where the three letters indicate the precision of the factorization, the working precision, and the precision with which residuals are computed, respectively. Various combinations of letters produce feasible algorithms (20 in all, if we include fixed precision refinement algorithms, such as “SSS”), of which HSD, HSQ, HDQ and SDQ use three different precisions. Similar results to those above apply to the latter three combinations.

Outlook

Our MATLAB experiments confirm the predictions of the error analysis regarding the behavior of Algorithm New and its GMRES-IR variant. They also show that the number of GMRES iterations in GMRES-IR can indeed be small.

Iterative refinement in three precisions therefore offers great promise for speeding up the solution of Ax = b. Instead of solving the system by an LU factorization at the working precision, we can factorize A at half the working precision and apply iterative refinement in three precisions, thereby obtaining a more accurate solution at potentially half the cost.

Full details of this work can be found in

Erin Carson and Nicholas J. Higham, Accelerating the Solution of Linear Systems by Iterative Refinement in Three Precisions MIMS EPrint 2017.24, Manchester Institute for Mathematical Sciences, The University of Manchester, UK, July 2017.

Posted in research | Leave a comment

Five Examples of Proofreading

proofs-authors-red.jpg

Every writer has also to be a proofreader, whether it be of his or her own drafts or of proofs sent by a publisher. In this post I will give some real-life examples of corrections to proofs. The problems to be corrected are not all errors: some are subtle aspects of the typesetting that need improvement. These examples should give you some ideas on what to look out for the next time you have a set of proofs to inspect.

Example 1

The first example is from proofs of one of my recent papers:

hist16-proof-line191.jpg

The article had been submitted as LaTeX source and it was reasonable to assume that the only differences between the proofs and what we submitted would be in places where a copy editor had imposed the journal style or had spotted a grammatical error. Fortunately, I know from experience not to make that assumption. These two sentences contain two errors introduced during copy editing: the term “Anderson acceleration” has been deleted after “To apply”, and “We denote by unvec” has been changed to “We denote by vec” (making the sentence nonsensical). The moral is never to assume that egregious errors have not been introduced: check everything in journal proofs.

In a similar vein, consider this extract from another set of proofs:

hino16-proof-line154.jpg

There is nothing wrong with the words or equations. The problem is that an unwanted paragraph break has been inserted after equation (2.6), and indeed also before “Only”. This set of proofs contained numerous unwanted added new paragraphs.

Example 2

Here is an extract from the proofs of my recent SIAM Review paper (with Natasa Strabic and Vedran Sego) Restoring Definiteness via Shrinking, with an Application to Correlation Matrices with a Fixed Block:

hss16-p4-proof.jpg

We noticed that the word “how” appears at the end of a line four times within seven lines—an unfortunate coincidence. We suggested that the production editor insert a hard space in the LaTeX source between one or more of the hows and the following word in order to force different line breaks. Here is the result as published:

hss16-p4-final.jpg

Example 3

What’s wrong with this example, from a paper in the 1980s?

besa89-p193.jpg

The phrase “best unknown” should be “best known”!

Example 4

The next example is from a book:

boyd14-p324.jpg

At first sight there is nothing wrong. But the 9z is suspicious: why 9, and why is this term that depends only on z inside the integral? It turns out that the equation should read

k(z) \equiv \frac{2}{z} \int_0^1 \tanh\bigl( z \sin(2\pi t) \bigr) \sin(2\pi t) \,dt.

When you realize that the left parenthesis and the digit 9 share the same key on the keyboard you can start to see how the error might have been made at the typing stage.

Example 5

The final example (from a 2013 issue of Private Eye) is completely different and illustrates a rare phenomenon:

private_eye_river_text1.jpg

If you cannot see anything wrong after a minute or so, click here. This phenomenon, whereby white spaces in successive lines join up to make a snake, is known as rivers of white. The fix, as in Example 2, is to force different line breaks.

Posted in writing | Leave a comment

SIAM Annual Meeting 2017 Preview

It’s a month to the 2017 SIAM Annual Meeting at the David Lawrence Convention Center in Pittsburgh. We’re returning to the location of the 2010 meeting. The meeting is co-chaired by Des Higham (University of Strathclyde) and Jennifer Mueller (Colorado State University).

Here are a few highlights and things it’s useful to know. If you haven’t already made plans to attend it’s not too late to register. Be sure to take in the view from the roof of the convention center, as shown here.

100716-2047-03-0774.jpg

Block Lecture by Emily Shuckburgh

The I. E. Block Community Lecture on Wednesday evening will be given by Emily Shuckburgh on From Flatland to Our Land: A Mathematician’s Journey through Our Changing Planet. Emily, from the British Antarctic Survey, is a co-author of the recent book Climate Change, which she wrote with HRH Prince Charles and Tony Juniper.

Prize Lectures

As always, a number of prize lectures will be given at the meeting. These include the four-yearly James H. Wilkinson Prize in Numerical Analysis and Scientific Computing, which will be awarded to Lek-Heng Lim. His lecture is titled Tensors in Computational Mathematics. See this article about Lek-Heng.

Joint with Activity Group Conferences and Workshops

The meeting is held jointly with the SIAM Conference on Industrial and Applied Geometry (GD17) and the SIAM Conference on Control and Its Applications (CT17), in the same location. One registration fee gains you access to all three meetings!

In addition, the SIAM Workshop on Parameter Space Dimension Reduction (DR17) and the SIAM Workshop on Network Science (NS17) are taking place just before and just after the conference, respectively.

Funding

Funding of mathematics, and other subjects, is in a state of uncertainty under the current US administration. In the minisymposium How Changing Implementations of National Priorities Might Affect Mathematical Funding a panel of representatives from funding agencies will describe the current situation and future opportunities. This is a great chance to hear the latest news from Washington from those in the know.

Students

SIAM provides a host of activities for students, beginning with an orientation session on Sunday evening and including a career fair, a session on career opportunities in business, industry and government (BIG), and the chance to meet and talk to invited speakers and co-chairs.

Hidden Figures

An evening session will include Christine Darden, who was one of the human computers included in the book “Hidden Figures” by Margot Lee Shetterly, on which the recent Hollywood movie of the same title was based.

SIAM Business Meeting

The Business Meeting (Tuesday at 6.15pm) provides an opportunity to hear the president (that’s me!) and SIAM staff report on SIAM’s activities over the past year and to ask questions. The 2017 SIAM Fellows will be recognized, and a reception in their honor follows the Business meeting.

Website

SIAM is developing a new website. A preliminary version will be available on laptops in the exhibit hall for participants to try. Feedback will be much appreciated and SIAM staff will be on hand to receive your comments.

Baseball Match

If you are staying in Pittsburgh on the Friday night, consider attending a baseball match. The Pittsburgh Pirates play the St Louis Cardinals at home at PNC Park on Friday July 14. I went to the Friday match after SIAM AN10 and really enjoyed it; the views from the ground are spectacular.

100716-2057-10-0196.jpg

Twitter

If you are not able to attend you can get a feel for what’s going on by following the hashtag #SIAMAN17 on Twitter.

Pittsburgh

There’s plenty to do and see in Pittsburgh, as the following images illustrate. As well as the impressive bridges over the Allegheny and Monongahela rivers, and some interesting downtown architecture and murals, there’s the Andy Warhol Museum (a short walk from the convention center). Here are some images I took in 2010.

100716-2057-19-0797.jpg 100716-1953-56-0749.jpg 100716-2013-14-0769.jpg 100716-1620-53-0547-Edit.jpg 100716-1800-58-0606.jpg 100716-1525-36-0533-Edit.jpg 100716-1524-16-0529.jpg 100716-1624-56_0551_Edit.jpg

Posted in conferences | Tagged | 2 Comments

A Second Course in Linear Algebra, by Garcia and Horn (2017)

gaho17-cover.jpg

The publication of a new linear algebra textbook is not normally a cause for excitement. However, Roger Horn is co-author of two of the most highly regarded and widely used books on matrix analysis: Matrix Analysis (2nd edition, 2013) and Topics in Matrix Analysis (1991), both co-authored with Charles Johnson. It is therefore to be expected that this new book by Garcia and Horn will offer something special.

Chapter 0 (Preliminaries) summarizes basic concepts and definitions, often stating results without proof (for example, properties of determinants). Chapters 1 (Vector Spaces) and 2 (Bases and Similarity) are described as reviews, but give results with proofs and examples. The second course proper starts with Chapter 3 (Block Matrices). As the chapter title suggests, the book makes systematic use of block matrices to simplify the treatment, and it is very much based on matrices rather than linear transformations.

Two things stand out about this book. First, it lies part-way between a traditional linear algebra text and texts with a numerical linear algebra focus. Thus it includes Householder matrices (but not Givens matrices), QR factorization, and Cholesky factorization. The construction given for QR factorization is essentially the Householder QR factorization, but the existence proof for Cholesky goes via the QR factor of the Hermitian positive definite square root, rather than by constructing the Cholesky factor explicitly via the usual recurrences. The existence of square roots of Hermitian positive definite matrices is proved via the spectral decomposition. It is possible to prove the existence of square roots without using the spectral theorem, and it would have been nice to mention this, at least in an exercise.

The second impressive aspect of the book is the wide, and often quite advanced, range of topics covered, which includes polar decomposition, interlacing results for the eigenvalues of Hermitian matrices, and circulant matrices. Not covered are, for example, Perron–Frobenius theory, the power method, and functions of nonsymmetric matrices (though various special cases are covered, such as the square root of Jordan block, often in the problems). New to me are the QS decomposition of a unitary matrix, Shoda’s theorem on commutators, and the Fuglede–Putnam theorem on normal matrices.

The 16-page index occupies 3.7 percent of the book, which, according to the length criteria discussed in my article A Call for Better Indexes, is unusually thorough. However, there is some over-indexing. For example, the entry permutation consists of 7 subentries all referring to page 10, but “permutation, 10” would have sufficed. An index entry “Cecil Sagehen” puzzled me. It has two page locators: one on which that term does not appear and one for a problem beginning “Cecil Sagehen is either happy or sad”. A little investigation revealed that “Cecil the Sagehen” is the mascot of Pomona College, which is the home institution of the first author.

There is a large collection of problems that go well beyond simple illustration and computation, and it is good to see that the problems are indexed.

Here are some other observations.

  • The singular value decomposition (SVD) is proved via the eigensystem of A^*A. Personally, I prefer the more elegant, if less intuitively obvious, proof in Golub and Van Loan’s Matrix Computations.
  • The treatment of Gershgorin’s theorem occupies six pages, but it omits the practically important result that if k discs form a connected region that is isolated from the other discs then that region contains precisely k eigenvalues.
  • The Cayley-Hamilton theorem is proved by using the Schur form. I would do it either via the minimal polynomial or the Jordan form, but these concepts are introduced only in later chapters.
  • Correlation matrices are mentioned in the preface, but do not appear in the book. They can make excellent examples.
  • The real Schur decomposition is not included, but rather just the special case for a real matrix having only real eigenvalues.
  • Matrix norms are not treated. The Frobenius norm is defined as an inner product norm and, unusually, the 2-norm is defined as the largest singular value of a matrix. There are no index entries for “matrix norm”, “norm, matrix”, “vector norm”, or “norm, vector”.
  • The pseudoinverse of a matrix is defined via the SVD. The Moore-Penrose conditions are not explicitly mentioned.
  • Three pages at the front summarize the notation and point to where terms are defined. Ironically, the oft-used notation M_n for an n \times n matrix, is not included.
  • Applications are mentioned only in passing. However, this does keep the book down to a relatively slim 426 pages.

Just as for numerical analysis texts, there will probably never exist a perfect linear algebra text.

The book is very well written and typeset. With its original presentation and choice of content it must be a strong contender for use on any second (or third) course on linear algebra. It can also serve as a reference on matrix theory: look here first and turn to Horn and Johnson if you don’t find what you want. Indeed a surprising amount of material from Horn and Johnson’s books is actually covered, albeit usually in less general form.

Posted in books | 1 Comment

SIAM Books Available Worldwide from Eurospan

eurospan.jpg

SIAM books are now available to individuals, bookstores, and other retailers outside North America from Eurospan, who have taken over the role previously carried out by Cambridge University Press.

This is significant news for those of us outside North America for two reasons.

  • Shipping is free, anywhere in the world.
  • SIAM members get a 30 percent discount on entering a special code at checkout. This code was emailed to all SIAM members in March 2017. If you are a SIAM member and do not have the code, you can get it by contacting SIAM customer service at siambooks@siam.org.

Check out the landing page for SIAM books at the Eurospan website.

SIAM members outside North America now have three options for ordering SIAM books.

  • Eurospan: 30 percent member discount; pay in GBP in the UK, euros in Europe, Australian dollars in Australian territories, US dollars everywhere else; free shipping.
  • SIAM Bookstore: 30 percent member discount; pay in US dollars; free shipping (at least for the near future).
  • Amazon: unpredictable, sometimes very small, discount; pay in currency of the particular site.
Posted in books | Tagged | Leave a comment

Dot Grid Paper for Writing Mathematics

As I discussed in Writing Mathematics in Pencil, and Why Analogue is Not Dead, there is a lot to be said for writing mathematics on paper, at least for early drafts before the material is typed into LaTeX.

There are essentially four types of paper that you might use.

  • Plain paper. Readily available: you can always raid the printer or photocopier. A plain sheet of paper places no constraints on your writing, but it can make it hard to maintain straight lines and a consistent letter height.
  • Ruled paper. The most popular choice. A drawback is that it may be hard to find the perfect line spacing, which can depend on what you are writing or even what pen you are using.
  • Graph, or quadrille, paper. Although aimed at those needing to draw graphs or designs, this paper can be used for general writing, as long as the lines are not so prominent as to be distracting.

There is a fourth type of paper that is less well known, but is becoming more popular: dot grid paper. This paper contains a rectangular array of dots. It is particularly popular with bullet journal enthusiasts.

Could dot grid paper be the perfect choice for writing mathematics? The dots are sufficient to keep your writing straight, but there is less ink on the page to distract you in the way that rules or a graph pattern can. If you need to draw a diagram or graph then the dots are most likely all the guidance you need. And you can draw boxes through groups of four dots to make a to-do list. As explained on the Baron Fig website, dot grid is “there when you need structure, quiet when you don’t”.

A popular supplier of dot grid paper is Rhodia, whose Dot Pads have lightly printed dots at 5mm intervals. The pads are stapled at the top, with the cover pre-scored in order to help it fold around the back of the pad. They also have micro-perforations that make it very easy to tear a page off. Their paper is much-loved by users of fountain pens for its smooth quality and resistance to bleed-through.

For general comments on dot grid paper from the online stationer Bureau Direct, some great photos, and even a flowchart (written on dot grid of course), see 3 Reasons To Switch To Dot Paper.

Here is a sample of mathematics written on Rhodia dot grid paper, using a Tombow Mono 100 4B pencil.

dotgrid-notes.jpg

Of course, you can generate your own dot grid paper with suitable LaTeX code. The following code is adapted from this post on Reddit; it produces this A4 sheet.

\documentclass{article}
\pagenumbering{gobble}
\usepackage[a4paper,hmargin={0mm,3mm},vmargin=5mm]{geometry}  
\usepackage{tikz}
\begin{document}
  \begin{tikzpicture}[scale=.5]
    \foreach \x in {0,...,41}
    \foreach \y in {0,...,57}
    {
  \fill[gray!75] (\x,\y) circle (0.06cm);
    }       
  \end{tikzpicture}
\end{document}

Give dot grid paper a try. It could be just what you need to unleash your mathematical (or other) creativity.

Posted in writing | Tagged | 2 Comments

Elements of MATLAB Style

140415-1443-12-4032-cropped.jpg

Style is an important aspect of writing, and also of programming. While MATLAB is a quick and easy language in which to program, style should not be neglected. Good style aids readability, which in turn makes it easier to debug and maintain code. It also fosters confidence in the code. Here are six style tips, in which the examples given are all inspired by real-life code that I have come across.

For more on MATLAB style see the book MATLAB Guide, and in particular section 16.1, “Elements of Coding Style”.

1. Omit Unnecessary Parentheses

% Bad style.
x = (A')\b;
y = [1];
D = (diag(x));

The parentheses around A' are unnecessary. Without them, the statement is legal and unambiguous. But of course the parentheses would be necessary in, for example, x = (A*B)'\c;

y is a scalar. There is no need to enter it as a 1-by-1 matrix by putting it in square brackets.

There is no need for parentheses around the diag function.

% Good style.
x = A'\b;
y = 1;
D = diag(x);

2. Use Spaces Consistently

% Bad style.
q=sqrt(2);
a = 1;
Z =zeros(3);
for i=1:10, q=q+1/q; end

Stick to a convention about spacing around around equals signs and plus and minus signs. I think spaces make the code more readable.

% Good style.
q = sqrt(2);
a = 1;
Z = zeros(3);
for i = 1:10, q = q + 1/q; end

3. Omit Unnecessary Semicolons and Commas

% Bad style.
figure;
for i = 2:n,
    x(i) = x(i-1)^2*y(i);
end;

Semicolons suppress output, but in this example there is no need for the them since no output is returned by the figure or end functions. The comma would only be necessary if the for loop was collapsed onto one line.

% Good style.
figure
for i = 2:n
    x(i) = x(i-1)^2*y(i);
end

4. Use the Appropriate Construct

The following if statement is perfectly correct.

% Bad style.
if flag == 1
   fprintf('Failed to converge.\n')
elseif flag == 2
   fprintf('Overflow encountered.\n')
elseif flag == 3
   fprintf('Division by zero.\n')
elseif flag == 4
   fprintf('Tolerance too small.\n')
end

But given its regular structure, it is better written as a switch statement, which is shorter, brings out the parallelism in the logic, and is easier to check.

% Good style.
switch flag
   case 1, fprintf('Failed to converge.\n')
   case 2, fprintf('Overflow encountered.\n')
   case 3, fprintf('Division by zero.\n')
   case 4, fprintf('Tolerance too small.\n')
end

5. Omit Trailing Tildes

In the list of output arguments in a function call a tilde signifies that the output argument in that position is not wanted and so can be discarded (and hence need not be computed). This notation was introduced in MATLAB R2009b. Generally, there is no point in providing a tilde as the last output argument, as a well written function will check the number of requested outputs and not compute any trailing outputs that are not requested.

The singular value decomposition (SVD) of a matrix A is computed by the call [U,S,V] = svd(A). In the following example we wish to compute the left singular vectors of A but not the singular values or right singular vectors.

% Bad style.
[U,~,~] = svd(A);

% Good style
[U,~] = svd(A);

An obvious question is why the second example was not shortened to

U = svd(A);

The reason is that with one output argument the svd function has a special behavior: it returns a matrix of singular values, not the matrix U of left singular vectors.

6. Include an H1 line in Program Files

The H1 line in a MATLAB program file is a comment line that is the second line in the file and contains the program name followed by a one-line description of the program. It is good practice to provide every program with an H1 line, as MATLAB itself does. Several MATLAB functions make use of H1 lines. For example, when the help function is invoked with a directory name and the directory in question does not contain a contents.m file, a list of contents is created from the H1 lines of the program files in the directory and displayed.

Here is an example of a function with an H1-line (this is an updated version of a function from The Matrix Computation Toolbox).

function show(x)
%SHOW   Display signs of matrix elements.
%   SHOW(X) displays X in `FORMAT +' form, that is,
%   with `+', `-' and  blank representing positive, negative
%   and zero elements respectively.

f = get(0,'Format'); % Get current numerical format.
format +
disp(x)
format(f)            % Restore original numeric format.
Posted in software | Tagged | 2 Comments

How to Print a Page Across Multiple Pages with Adobe Acrobat

Occasionally I need to proof a PDF document that is too small to read comfortably when printed in the usual way. This is the case with my columns for SIAM News, as SIAM News is A3 format whereas my printer is A4.` ‘ A simple solution is to use the Poster option under “Page Size & Handling” in Adobe Acrobat’s print dialog box. Acrobat will print each page over multiple pages that, if joined together, reproduce the original pages (except for the white margins). The preview window in the dialog box shows how each page will be split into pieces. You can adjust the percentage in the “Tile Scale” box to increase or decrease the number of printed pages per original page.

acrobat-poster.jpg

(This screen capture shows Adobe Acrobat version 11.0.19; the dialog box may be different in other versions of Acrobat.)

Here is a photograph of the result laid out on my office floor, with the four A4 sheets arranged to match the original document. (This is the proofs of my April 2017 SIAM News column.)

170317-0952-56-6777.jpg

This Poster option is no doubt particularly intended for proofing posters on an A4 or A3 printer, as they are typically designed for printing at A0 or A1.

You may notice that in the screen capture above my printer is Fineprint. This is an application that acts as a Windows printer driver and captures and displays what is printed to it, possibly from multiple print commands. It allows the user to reorder, magnify (via the “Enlarge” option), and edit pages before actually printing them, and printing can be 1-up, 2-up, 4-up, or 8-up. I have been using FinePrint for many years and could not live without it. It saves me paper and by allowing me to reduce the margins makes documents much easier to read. The following screen capture shows FinePrint after printing the document shown above with the Poster option at 120%.

fineprint-ex.jpg

Posted in software | Tagged , | Leave a comment

What’s New in MATLAB R2017a?

MATLAB R2017a was released last week. Many of the changes reported in the release notes are evolutionary, building on and extending major new features introduced previously. For example, the Live Editor continues to gain expanded capabilities. In this post I pick out a few new features that caught my eye. This is very much a personal selection. For full details of what’s new, see the release notes, and see also my previous post What’s New in MATLAB R2016b if you are not familiar with R2016b.

Parula

The parula color map has been modified slightly. The difference is subtle, but as the following example illustrates, the R2017a parula is a bit more vibrant and has a bit less cyan and yellow in the blues and greens.

parula-2016b-2017a.jpg

I note, however, that the difference between the old and the new parula is smaller when the images are converted to the CMYK color space, as they must be for printing.

Heatmap

The new heatmap function plots a heatmap of tabular data. Although it is intended mainly for use with the table data type, I think heatmap will be useful for getting insight into the structure of matrices, as illustrated by the following examples.

heatmaps.jpg

String Arrays

String arrays, introduced in MATLAB R2016b, can now be formed using double quotes:

>> s = string('This is a string') % R2016b
s = 
    "This is a string"
>> t = "This is a string"         % R2017a
t = 
    "This is a string"
>> isequal(s,t)
ans =
  logical
   1
>> whos
  Name      Size            Bytes  Class     Attributes

  s         1x1               166  string              
  t         1x1               166  string

However, there is one major caveat. Many MATLAB functions that take a char as input argument have not yet been adapted to accept strings. Hence

>> A = gallery('moler',3)
A =
     1    -1    -1
    -1     2     0
    -1     0     3
>> A = gallery("moler",3)
Error using nargin
Argument must be either a character vector or a function handle.
Error in gallery (line 191)
nargs = nargin(matname);

I expect that such functions will be updated in future releases.

Missing

A new function missing creates missing values appropriate to the data type in question.

>> A = ones(2); A(2,2) = missing
A =
     1     1
     1   NaN

> d = datetime('2014-05-26')
d = 
  datetime
   26-May-2014
>> d(2) = missing
d = 
  1×2 datetime array
   26-May-2014   NaT

Until converted to the target datatype, a missing value has class missing:

>> m = missing
m = 
  missing
    
>> class(m)
ans =
    'missing'

Performance Improvements

The release notes report performance improvements under a variety of headings, including execution engine, scripts, and mathematics functions. These are very welcome, as the user automatically benefits from them. One comment that caught my eye is “The backslash command A\B is faster when operating on negative definite matrices”. I think this means that MATLAB checks whether the matrix, A, is symmetric with all negative diagonal elements and, if it is, attempts a Cholesky factorization of -A.

Posted in software | Tagged | Leave a comment

Tracing the Early History of MATLAB Through SIAM News

A recent blog post by Ned Gulley points out that the new mathematics gallery (“Mathematics: The Winton Gallery”) at the Science Museum, London, contains a copy of the disk and manual for MATLAB 1.3, from 1985, sitting next to a trial assembly of Charles Babbage’s analytical engine.

This set me thinking about the early history of MATLAB and The MathWorks. Items such as manuals and disks must be quite rare nowadays. What other traces are there of early MATLAB history? I have recently been looking through some back issues of SIAM News and spotted a number of adverts for MATLAB over the period 1985–1991. Let’s see what historical insight these early adverts give.

1985

sinews-1985-18-2-mathworks_ad.jpg PDF file

The first advert I can find is from the March 1985 SIAM News, and it is for PC-MATLAB, priced at $695, running on an IBM-PC or compatible computer. The advert features the now famous MathWorks logo, which represents an eigenfunction of the L-shaped membrane. It quotes a time of 10.1 seconds for a 50-by-50 real matrix multiplication on a machine with an Intel 8087 coprocessor. This floating-point coprocessor was a useful add-on to the IBM-PC, which used the Intel 8086 chip.

MATLAB benefited from being launched at a time when the IBM PC with 8087 coprocessor was just starting to become popular. The 8086-8087 combination made it possible to carry out computations on a desktop PC that had previously required a minicomputer—and they could now be done with the interactive MATLAB interface.

The advert mentions “mainframe MATLAB”, which it says is written in Fortran, runs on “larger computers”, and is in use in several hundred organizations worldwide.

PC-MATLAB had been rewritten in C, and it supported graphics, IEEE arithmetic (as implemented in the 8087), and “user-defined functions” (M-files).

Note that at this time MathWorks was located at 124 Foxwood Road, Portola Valley, California. In his article The Growth of MATLAB and The MathWorks Over Two Decades, Cleve Moler explains that this first mailing address was “a rented A-frame cabin where Jack [Little] lived in the hills above Stanford University in Portola Valley, California”.

The September 1985 issue of SIAM News contains a rather different advert that now refers to “the original mainframe version of MATLAB” and emphasizes the ease of use of MATLAB.

sinews-1985-18-5-mathworks_ad.jpg PDF file

1987

sinews-1987-20-1-mathworks_ad.jpg PDF file

The next advert is from the January 1987 SIAM News. MATLAB is now also available as Pro-MATLAB running on a Sun workstation or a VAX computer. M-files are now mentioned by that name and LINPACK benchmark figures are stated, the largest figure being 98 Kflops on the MicroVAX II.

The MathWorks has now moved to Massachusetts.

1990

sinews-1990-23-3-mathworks_ad.jpg PDF file

The advert in the May 1990 SIAM News gives a version number, MATLAB 3.5, and it announces the Signal Processing Toolbox. It boasts that MATLAB has over 400 built-in functions and supports an enlarged range of computers, which include Cray supercomputers. The benchmark for a 50-by-50 real matrix multiplication is now down to 0.71 seconds on a 20 Mhz 386-based PC: a reduction by a factor of 14 from the figure quoted in 1985.

A strange feature of the advert is that the toolbox is only explicitly mentioned in the box at top left and the meaning of “toolbox” is not stated.

The MathWorks address has changed again, to 21 Eliot St., South Natick, Massachusetts. In the article mentioned above, Cleve explains, “When the company reached about a dozen employees, we moved several miles east to take over the second floor of a lovely building in South Natick, Massachusetts”.

1991

sinews-1991-24-5-mathworks_ad.jpg PDF file

The advert in the May 1991 issue of SIAM News focuses on two new toolboxes: the Spline Toolbox and the Optimization Toolbox. The gray box defines toolboxes as “sets of routines written in the MATLAB programming language for specialized applications”.

MathWorks is now located at Cochituate Place on Prime Park Way in Natick. The street was so-named because it had been the home of the Prime Computer Corporation, which produced minicomputers from 1972 to 1992.

These adverts give some insight into the development of MATLAB in its early years. They also show how the rapid growth of MathWorks necessitated frequent relocation of the company. Indeed, in the article mentioned above Cleve Moler notes that the number of employees roughly doubled every year for the first seven years.

Posted in software | Tagged , | Leave a comment