All the links below are free high level languages and Mathematics Packages for Scientific Computation on Linux. These packages are usually like a Mathematical Laboratory in which numerical computations can be done and usually have their own interpreted language. They either link to a popular (free) plotting package or have their own graphics and plotting capability. They also provide capability to I/O files and interface with other programming languages like C, C++, Fortran, etc ... Now a days some of them have parallel programming capabilities. I have not included MuPAD, a good symbolic math package, since is not really free. Check out if their most free license suits you.
Octave: An excellent package for numerical computations. It uses gnuplot for plotting and has a online help. It is also easily extensible (i.e. new functions, procedures can be written) either using its own language or by using dynamically loadable modules written in C, C++, Fortran or other languages. An extensive manual is available here. You can get a GNOME based front end for it here. It is distributed under the GNU Public License.
Scilab: Another superb package numerical computations having a good user interface and a very good online click-able help. Its plotting and graphic capabilities are also very impressive. It also provides for easy interfacing with Fortran and C. It has its own free license.
Yorick: Yorick is a fast, interpreted language, designed for scientific computing and numerical analysis. The syntax is similar to C, but the variables need not be declared. It offers an interactive graphics package based on X windows. X-Y plots, quadrilateral meshes, filled meshes, cell arrays, and contours are supported. You can embed compiled routines in Yorick to solve problems for which the interpreter is too slow. It is also useful as a pre and post processor for large physical simulation programs. A tutorial like manual is available here. Yorick is open source software, copyright of the Regents of the University of California.
Algae: As the above link describes it, Algae is a interpreted language for numerical analysis. It was developed as a fast and versatile tool, capable of handling large problems. Algae consists of the programming language Algae, and algae, the interpreter. Its features include speed (generally much faster than octave, RLaB and Scilab), storage of sparse arrays and a code profiling capability (to check where your code spends its time). A user manual is available here. It is distributed under the GNU General Public License.
YACAS: As the above link describes it, "YACAS is an easy to use, general purpose Computer Algebra System, a program for symbolic manipulation of mathematical expressions. It uses its own programming language designed for symbolic as well as arbitrary-precision numerical computations". Links to documentation (user manual, tutorial, etc ..) is available here. It is distributed under the GNU General Public License.
RLAB: The above link describes it thus, "Rlab is an interactive, interpreted scientific programming environment. Rlab is a very high level language intended to provide fast prototyping and program development, as well as easy data-visualization, and processing". It is distributed under the GNU General Public License. The author Ian Searle has written an article in The Linux Journal titled An Introduction to Rlab which as he reminds us, is a bit dated, and a Reference Manual is also available.
Maxima: Maxima is a symbolic computation program. As the link above describes it, "Maxima is a Common Lisp implementation of MIT's Macsyma system for computer based algebra". George White has maintained and extended Maxima for the last 18 years, but has formal permission from DOE, to release this under GPL as a derivative work science 1998. I am looking forward to installing this and yacas above to check out their symbolic computation capabilities.
The R-Project for Statistical Computing: R is a language and environment for statistical computing and graphics. It provides a large collection of tools for statistical analysis of large arrays of data and also graphical facilities. R is also a complete effective programming language. For computationally intensive tasks, C, C++ and Fortran code can be linked and called at run time. A comprehensive set of manuals dealing with installation, introduction, writing extensions, etc ... is available here. It is distributed under the GNU General Public License.
gTybalt: gTybalt is a step towards a free computer algebra system. It is object oriented, allowing symbolic calculations within C++. It is efficient, in the sense that solutions developed with gTybalt can be compiled with a C++ compiler and executed independently of gTybalt. The mathematical formulae are visualized using TeX fonts and can easily be converted to LaTeX. I did not realize that it has good graphic capabilities till I checked out the gTybalt manual. It is distributed under the GNU General Public License.
JACAL: As the link above describes it, " JACAL is an interactive symbolic mathematics program. JACAL can manipulate and simplify equations, scalars, vectors, and matrices of single and multiple valued algebraic expressions containing numbers, variables, radicals, and algebraic differential, and holonomic functions".
bc: bc is an arbitrary precision numeric processing language. It supports interactive execution of statements. Click here for a Manual in a variety of formats. It is GNU software and is distributed under the GNU General Public License.