[4122] | 1 | """Collection of Fortran 90 namelist helper functions. |
---|
| 2 | |
---|
| 3 | A common way to interface with a Fortran executable is via |
---|
| 4 | an input file called a namelist. This module defines |
---|
| 5 | functions which simplify the process of updating and |
---|
| 6 | extending namelist data. |
---|
| 7 | |
---|
| 8 | .. note:: This module is especially lightweight and follows the |
---|
| 9 | batteries included philosophy. As such, only standard |
---|
| 10 | library modules are required to use this code. |
---|
| 11 | |
---|
| 12 | Walkthrough |
---|
| 13 | =========== |
---|
| 14 | |
---|
| 15 | New namelist |
---|
| 16 | ------------ |
---|
| 17 | |
---|
| 18 | A typical usage is to create and update a Fortran namelist on the fly. |
---|
| 19 | |
---|
| 20 | >>> import nml |
---|
| 21 | >>> namid = "namfoo" |
---|
| 22 | >>> text = nml.new(namid) |
---|
| 23 | >>> data = {"x": nml.tostring([1, 2, 3])} |
---|
| 24 | >>> text = nml.update(namid, text, data) |
---|
| 25 | >>> print text |
---|
| 26 | &namfoo |
---|
| 27 | x = 1 2 3 |
---|
| 28 | / |
---|
[4131] | 29 | <BLANKLINE> |
---|
[4122] | 30 | |
---|
| 31 | In the above snippet :func:`tostring` has been used to sanitize the input |
---|
| 32 | Python list. This function cleverly maps string data and numeric data to |
---|
| 33 | the correct Fortran syntax. |
---|
| 34 | |
---|
| 35 | However, the :func:`new` function takes care of many of the above steps automatically. |
---|
| 36 | Where appropriate :func:`sanitize` has been embedded to reduce the need |
---|
| 37 | to worry about data format problems. Take for example, |
---|
| 38 | |
---|
| 39 | >>> print nml.new("namfoo", data={"x": range(3)}) |
---|
| 40 | &namfoo |
---|
| 41 | x = 0 1 2 |
---|
| 42 | / |
---|
[4131] | 43 | <BLANKLINE> |
---|
[4122] | 44 | |
---|
| 45 | Parse existing namelist |
---|
| 46 | ----------------------- |
---|
| 47 | |
---|
| 48 | In order to update a namelist it is necessary to convert the namelist text into |
---|
| 49 | a dictionary of *key, value* pairs which can be manipulated in the usual Pythonic |
---|
| 50 | fashion before being piped back out to disk. |
---|
| 51 | |
---|
| 52 | In everyday usage text will be read from files, here however for illustration |
---|
| 53 | purposes I have hand written a namelist. |
---|
| 54 | |
---|
| 55 | >>> text = ''' |
---|
| 56 | ... &namfoo |
---|
| 57 | ... x = y ! A description of the variables |
---|
| 58 | ... / |
---|
| 59 | ... &nambar |
---|
| 60 | ... ln_on = .TRUE. ! A description of the variables |
---|
| 61 | ... / |
---|
| 62 | ... ''' |
---|
| 63 | |
---|
| 64 | This can be parsed by invoking the :func:`variables` command. |
---|
| 65 | |
---|
| 66 | >>> nml.variables(text) |
---|
| 67 | {'x': 'y', 'ln_on': '.TRUE.'} |
---|
| 68 | |
---|
| 69 | Or by using the :func:`namelists` function to split the file into sub-lists. |
---|
| 70 | |
---|
| 71 | >>> nml.namelists(text) |
---|
| 72 | {'namfoo': '&namfoo\\n x = y ! A description of the variables\\n/', 'nambar': '&nambar\\n ln_on = .TRUE. ! A description of the variables\\n/'} |
---|
| 73 | >>> sublists = nml.namelists(text) |
---|
| 74 | >>> print sublists["nambar"] |
---|
| 75 | &nambar |
---|
| 76 | ln_on = .TRUE. ! A description of the variables |
---|
| 77 | / |
---|
| 78 | |
---|
| 79 | Which can be parsed into a dictionary as before. |
---|
| 80 | |
---|
| 81 | >>> print nml.variables(sublists["nambar"]) |
---|
| 82 | {'ln_on': '.TRUE.'} |
---|
| 83 | |
---|
| 84 | |
---|
| 85 | Update/replace data |
---|
| 86 | ------------------- |
---|
| 87 | |
---|
| 88 | There are two ways of modifying values inside a Fortran namelist. |
---|
| 89 | |
---|
| 90 | Replace |
---|
| 91 | The first is to simply replace a set of variables with new values. This behaviour is accomplished |
---|
| 92 | via the :func:`replace` function. This approach simply overwrites existing variables. No knowledge |
---|
| 93 | of sub-namelist structure is required to modify a string of text. |
---|
| 94 | |
---|
| 95 | .. note:: Additional variables will not be added to a namelist via this approach |
---|
| 96 | |
---|
| 97 | Update |
---|
| 98 | The second is to extend the set of variables contained within a namelist. This functionality is |
---|
| 99 | controlled by the :func:`update` function. Here, variables which are not already specified are |
---|
| 100 | added using a templated namelist line. |
---|
| 101 | |
---|
| 102 | .. note:: It is essential to specify which sub-namelist is to be updated before modification takes place |
---|
| 103 | |
---|
| 104 | Pipe to/from file |
---|
| 105 | ----------------- |
---|
| 106 | |
---|
| 107 | As typical NEMO namelists are no larger than a few tens of kilobytes |
---|
| 108 | it makes sense to process namelists as single strings instead of |
---|
| 109 | line by line. |
---|
| 110 | |
---|
| 111 | >>> path = "foo.nml" |
---|
| 112 | >>> text = nml.new("namfoo") |
---|
| 113 | |
---|
| 114 | To write to a file simply invoke the writer. |
---|
| 115 | |
---|
| 116 | >>> # Write to file |
---|
| 117 | >>> nml.writer(path, text) |
---|
| 118 | |
---|
| 119 | To read from a file specify the path to be read. |
---|
| 120 | |
---|
| 121 | >>> # Read from file |
---|
| 122 | >>> text = nml.reader(path) |
---|
| 123 | |
---|
| 124 | Join multiple namelists |
---|
| 125 | ----------------------- |
---|
| 126 | |
---|
| 127 | Since the namelists are regular Python strings there is no need for a |
---|
| 128 | specific *join* function. Namelists can be combined in whatever manner |
---|
| 129 | is most pleasing to the eye. |
---|
| 130 | |
---|
| 131 | >>> namoff = nml.new("namoff") |
---|
| 132 | >>> namcl4 = nml.new("namcl4") |
---|
| 133 | >>> # new line join |
---|
| 134 | >>> print "\\n".join([namoff, namcl4]) |
---|
| 135 | &namoff |
---|
| 136 | / |
---|
| 137 | <BLANKLINE> |
---|
| 138 | &namcl4 |
---|
| 139 | / |
---|
[4131] | 140 | <BLANKLINE> |
---|
[4122] | 141 | |
---|
| 142 | >>> # Or addition |
---|
| 143 | >>> print namoff + namcl4 |
---|
| 144 | &namoff |
---|
| 145 | / |
---|
| 146 | &namcl4 |
---|
| 147 | / |
---|
[4131] | 148 | <BLANKLINE> |
---|
[4122] | 149 | |
---|
| 150 | Module functions |
---|
| 151 | ================ |
---|
| 152 | |
---|
| 153 | """ |
---|
| 154 | __version__ = "0.1.0" |
---|
| 155 | import re |
---|
[4128] | 156 | from numbers import Number |
---|
[4122] | 157 | |
---|
| 158 | def reader(path): |
---|
| 159 | """Reads a file into a string |
---|
| 160 | |
---|
| 161 | Reads whole file into single string. Typically, |
---|
| 162 | namelists are small enough to be stored in memory |
---|
| 163 | while updates and edits are being performed. |
---|
| 164 | |
---|
| 165 | :param path: Path to input file |
---|
| 166 | :returns: entire file as a single string |
---|
| 167 | |
---|
| 168 | """ |
---|
| 169 | with open(path, "r") as handle: |
---|
| 170 | text = handle.read() |
---|
| 171 | return text |
---|
| 172 | |
---|
| 173 | def writer(path, text): |
---|
| 174 | """Writes to a file from a string |
---|
| 175 | |
---|
| 176 | Handy way of piping a processed namelist into |
---|
| 177 | a file. |
---|
| 178 | |
---|
| 179 | :param path: Path to output file |
---|
| 180 | :param text: Input text to process |
---|
| 181 | |
---|
| 182 | """ |
---|
| 183 | with open(path, "w") as handle: |
---|
| 184 | handle.write(text) |
---|
| 185 | |
---|
| 186 | def update(namid, text, data, convert=True): |
---|
| 187 | """Extends namelist definition. |
---|
| 188 | |
---|
| 189 | Similar to replace this function alters the values |
---|
| 190 | of variables defined within a namelist. In addition to |
---|
| 191 | replacing values it also creates definitions if the |
---|
| 192 | variable is not found in the namelist. As such, the |
---|
| 193 | namelist id must be specified. |
---|
| 194 | |
---|
| 195 | :param namid: Namelist id |
---|
| 196 | :param text: Input text to process |
---|
| 197 | :param data: Dictionary of variables |
---|
| 198 | :keyword convert: Sanitizes input data before replacement takes place |
---|
| 199 | |
---|
| 200 | :returns: Text |
---|
| 201 | |
---|
| 202 | .. seealso:: :func:`replace` :func:`sanitize` |
---|
| 203 | """ |
---|
| 204 | sublists = namelists(text) |
---|
| 205 | assert namid in sublists, "Warning: invalid namid specified!" |
---|
| 206 | |
---|
| 207 | # Sanitize inputs |
---|
| 208 | if convert: |
---|
| 209 | data = sanitize(data) |
---|
| 210 | |
---|
| 211 | # Parse subsection |
---|
| 212 | namtext = sublists[namid] |
---|
| 213 | subdata = variables(namtext) |
---|
| 214 | subvars = subdata.keys() |
---|
| 215 | |
---|
| 216 | # Replace existing variables in namtext |
---|
| 217 | tmptext = replace(namtext, data) |
---|
| 218 | text = text.replace(namtext, tmptext) |
---|
| 219 | namtext = tmptext |
---|
| 220 | |
---|
| 221 | # Identify new variables |
---|
| 222 | vars = data.keys() |
---|
| 223 | newvars = list(set(vars) - set(subvars)) |
---|
| 224 | newvars.sort() |
---|
| 225 | |
---|
| 226 | # Append new vars to namid |
---|
| 227 | lines = namtext.split("\n") |
---|
| 228 | for v in newvars: |
---|
| 229 | newline = " %s = %s" % (v, data[v]) |
---|
| 230 | lines.insert(-1, newline) |
---|
| 231 | newtext = "\n".join(lines) |
---|
| 232 | |
---|
| 233 | # Replace old namtext with new namtext |
---|
| 234 | text = text.replace(namtext, newtext) |
---|
| 235 | return text |
---|
| 236 | |
---|
| 237 | def replace(text, data, convert=True): |
---|
| 238 | """Edits existing variables. |
---|
| 239 | |
---|
| 240 | Pattern matches and substitutes variables inside |
---|
| 241 | a string of text. This is independent of namid and |
---|
| 242 | as such is useful for modifying existing variables. |
---|
| 243 | To append new variables the :func:`update` function |
---|
| 244 | is required. |
---|
| 245 | |
---|
| 246 | >>> text = ''' |
---|
| 247 | ... &namobs |
---|
| 248 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
| 249 | ... / |
---|
| 250 | ... ''' |
---|
| 251 | >>> data = {"ln_sst": ".FALSE."} |
---|
| 252 | >>> print replace(text, data) |
---|
| 253 | <BLANKLINE> |
---|
| 254 | &namobs |
---|
| 255 | ln_sst = .FALSE. ! Logical switch for SST observations |
---|
| 256 | / |
---|
| 257 | <BLANKLINE> |
---|
| 258 | |
---|
| 259 | .. note :: This does not append new variables to a namelist |
---|
| 260 | |
---|
| 261 | :param text: string to process |
---|
| 262 | :param data: dictionary with which to modify **text** |
---|
| 263 | :keyword convert: Sanitizes input data before replacement takes place |
---|
| 264 | |
---|
| 265 | :returns: string with new data values |
---|
| 266 | |
---|
| 267 | .. seealso:: :func:`update`, :func:`sanitize` |
---|
| 268 | """ |
---|
| 269 | if convert: |
---|
| 270 | data = sanitize(data) |
---|
| 271 | for k, v in data.iteritems(): |
---|
[4126] | 272 | pat = r"(%s\s*=\s*).+?(\s*[!\n])" % (k,) |
---|
[4122] | 273 | repl = r"\g<1>%s\g<2>" % (v,) |
---|
| 274 | text = re.sub(pat, repl, text) |
---|
| 275 | return text |
---|
| 276 | |
---|
| 277 | def variables(text): |
---|
| 278 | """Retrieves dictionary of variables in text. |
---|
| 279 | |
---|
| 280 | >>> text = ''' |
---|
| 281 | ... &namobs |
---|
| 282 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
| 283 | ... / |
---|
| 284 | ... ''' |
---|
| 285 | >>> variables(text) |
---|
| 286 | {'ln_sst': '.TRUE.'} |
---|
| 287 | |
---|
| 288 | :param text: Input text to process |
---|
| 289 | |
---|
| 290 | :returns: A dictionary of variable, value pairs. |
---|
| 291 | |
---|
| 292 | """ |
---|
| 293 | data = {} |
---|
| 294 | pairs = re.findall(r"\n\s*(\w+)\s*=\s*(.+?)\s*(?=[!\n])", text) |
---|
| 295 | for key, value in pairs: |
---|
| 296 | data[key] = value |
---|
| 297 | return data |
---|
| 298 | |
---|
| 299 | def namelists(text): |
---|
| 300 | """Retrieves dictionary of namelists in text. |
---|
| 301 | |
---|
| 302 | Useful for isolating sub-namelists. |
---|
| 303 | |
---|
| 304 | >>> text = ''' |
---|
| 305 | ... &namobs |
---|
| 306 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
| 307 | ... / |
---|
| 308 | ... ''' |
---|
| 309 | >>> namelists(text) |
---|
| 310 | {'namobs': '&namobs\\n ln_sst = .TRUE. ! Logical switch for SST observations\\n/'} |
---|
| 311 | |
---|
| 312 | :param text: Input text to process |
---|
| 313 | |
---|
| 314 | :returns: A dictionary of id, text block key, value pairs |
---|
| 315 | |
---|
| 316 | """ |
---|
| 317 | # Boundary case |
---|
| 318 | if text.startswith("&"): |
---|
| 319 | text = "\n" + text |
---|
| 320 | # Regular expression |
---|
| 321 | results = re.findall(r"\n(&(\w+).*?\n/)", text, re.DOTALL) |
---|
| 322 | data = {} |
---|
| 323 | for content, namid in results: |
---|
| 324 | data[namid] = content |
---|
| 325 | return data |
---|
| 326 | |
---|
| 327 | def tostring(data): |
---|
| 328 | """Maps standard Python data to Fortran namelist format. |
---|
| 329 | |
---|
| 330 | >>> tostring([1, 2, 3]) |
---|
| 331 | '1 2 3' |
---|
| 332 | >>> tostring(["foo.nc", "bar.nc"]) |
---|
[4129] | 333 | "'foo.nc', 'bar.nc'" |
---|
[4122] | 334 | >>> tostring(True) |
---|
| 335 | '.TRUE.' |
---|
| 336 | |
---|
| 337 | :param data: Input Python data |
---|
| 338 | |
---|
| 339 | :returns: Namelist formatted string |
---|
| 340 | |
---|
| 341 | .. seealso:: :func:`sanitize` |
---|
| 342 | """ |
---|
| 343 | if isinstance(data, list): |
---|
[4128] | 344 | if all_numeric(data): |
---|
[4126] | 345 | delim = " " |
---|
[4122] | 346 | else: |
---|
[4126] | 347 | delim = ", " |
---|
| 348 | text = delim.join([convert(item) for item in data]) |
---|
[4122] | 349 | else: |
---|
[4126] | 350 | text = convert(data) |
---|
[4122] | 351 | return text |
---|
| 352 | |
---|
[4128] | 353 | def all_numeric(inputs): |
---|
| 354 | # Checks all list entries are numbers |
---|
| 355 | flag = True |
---|
| 356 | for input in inputs: |
---|
| 357 | if not isinstance(input, Number): |
---|
| 358 | flag = False |
---|
| 359 | break |
---|
| 360 | return flag |
---|
| 361 | |
---|
[4126] | 362 | def numeric(word): |
---|
| 363 | # Tests input string is numeric data |
---|
| 364 | parts = word.split(" ") |
---|
| 365 | try: |
---|
| 366 | map(float, parts) |
---|
| 367 | flag = True |
---|
| 368 | except ValueError: |
---|
| 369 | flag = False |
---|
| 370 | return flag |
---|
| 371 | |
---|
| 372 | def logical(word): |
---|
| 373 | # Tests input string is numeric data |
---|
| 374 | if word.upper() in [".FALSE.", ".TRUE."]: |
---|
| 375 | flag = True |
---|
| 376 | else: |
---|
| 377 | flag = False |
---|
| 378 | return flag |
---|
| 379 | |
---|
| 380 | def listed(word): |
---|
| 381 | # Tests input string is not a list |
---|
| 382 | if ("," in word) or (" " in word): |
---|
| 383 | flag = True |
---|
| 384 | else: |
---|
| 385 | flag = False |
---|
| 386 | return flag |
---|
| 387 | |
---|
[4128] | 388 | def quote(word): |
---|
[4133] | 389 | word = str(word) |
---|
[4128] | 390 | if not quoted(word): |
---|
| 391 | word = "'%s'" % (word,) |
---|
| 392 | return word |
---|
| 393 | |
---|
[4126] | 394 | def convert(word): |
---|
| 395 | # Conversion function |
---|
| 396 | if isinstance(word, str): |
---|
| 397 | if (quoted(word) or numeric(word) |
---|
| 398 | or logical(word) or listed(word)): |
---|
| 399 | result = "%s" % (word,) |
---|
| 400 | else: |
---|
[4128] | 401 | result = quote(word) |
---|
[4126] | 402 | elif isinstance(word, bool): |
---|
| 403 | if word: |
---|
| 404 | result = ".TRUE." |
---|
| 405 | else: |
---|
| 406 | result = ".FALSE." |
---|
| 407 | else: |
---|
| 408 | result = str(word) |
---|
| 409 | return result |
---|
| 410 | |
---|
| 411 | def quoted(word): |
---|
| 412 | # Checks if string begins/ends with quotation marks |
---|
| 413 | if (word.startswith("'") and word.endswith("'")): |
---|
| 414 | flag = True |
---|
| 415 | elif (word.startswith('"') and word.endswith('"')): |
---|
| 416 | flag = True |
---|
| 417 | else: |
---|
| 418 | flag = False |
---|
| 419 | return flag |
---|
| 420 | |
---|
| 421 | def same_type(data): |
---|
| 422 | # True if all entries are the same type |
---|
| 423 | types = map(type, data) |
---|
| 424 | if len(set(types)) == 1: |
---|
| 425 | flag = True |
---|
| 426 | else: |
---|
| 427 | flag = False |
---|
| 428 | return flag |
---|
| 429 | |
---|
[4122] | 430 | def sanitize(data): |
---|
| 431 | """Converts dictionary values into Fortran namelist format. |
---|
| 432 | |
---|
| 433 | This is a more typical way to prepare data for inclusion in |
---|
| 434 | a Fortran namelist. Instead of manually applying :func:`tostring` |
---|
| 435 | to every element of the input data, **sanitize** fixes the entire |
---|
| 436 | data set. |
---|
| 437 | |
---|
| 438 | >>> sanitize({"x": True}) |
---|
| 439 | {'x': '.TRUE.'} |
---|
| 440 | >>> |
---|
| 441 | |
---|
| 442 | :param data: Dictionary to convert |
---|
| 443 | |
---|
| 444 | :returns: Dictionary whose values are in Fortran namelist format |
---|
| 445 | |
---|
| 446 | .. seealso:: :func:`tostring` |
---|
| 447 | """ |
---|
| 448 | replacements = [(k, tostring(v)) for k, v in data.items()] |
---|
| 449 | data.update(replacements) |
---|
| 450 | return data |
---|
| 451 | |
---|
| 452 | def new(namid, data=None, convert=True): |
---|
| 453 | """Creates a new Fortran namelist |
---|
| 454 | |
---|
| 455 | >>> new("namobs") |
---|
[4131] | 456 | '&namobs\\n/\\n' |
---|
[4122] | 457 | >>> print new("namobs") |
---|
| 458 | &namobs |
---|
| 459 | / |
---|
[4131] | 460 | <BLANKLINE> |
---|
[4122] | 461 | |
---|
| 462 | :param namid: Name for the new namelist |
---|
| 463 | |
---|
| 464 | :keyword data: Specifies an initial dictionary with which to |
---|
| 465 | populate the namelist |
---|
| 466 | :type data: dict |
---|
| 467 | :keyword convert: Sanitizes input data before replacement takes place |
---|
| 468 | |
---|
| 469 | :returns: string representation of a Fortran namelist |
---|
| 470 | """ |
---|
[4131] | 471 | text = "&{namid}\n/\n".format(namid=namid) |
---|
[4122] | 472 | if data is not None: |
---|
| 473 | text = update(namid, text, data, convert=convert) |
---|
| 474 | return text |
---|
| 475 | |
---|
| 476 | if __name__ == '__main__': |
---|
| 477 | import doctest |
---|
| 478 | doctest.testmod() |
---|
| 479 | |
---|