1 | """Collection of Fortran 90 namelist helper functions. |
---|
2 | |
---|
3 | A common way to interface with a Fortran executable is via |
---|
4 | an input file called a namelist. This module defines |
---|
5 | functions which simplify the process of updating and |
---|
6 | extending namelist data. |
---|
7 | |
---|
8 | .. note:: This module is especially lightweight and follows the |
---|
9 | batteries included philosophy. As such, only standard |
---|
10 | library modules are required to use this code. |
---|
11 | |
---|
12 | Walkthrough |
---|
13 | =========== |
---|
14 | |
---|
15 | New namelist |
---|
16 | ------------ |
---|
17 | |
---|
18 | A typical usage is to create and update a Fortran namelist on the fly. |
---|
19 | |
---|
20 | >>> import nml |
---|
21 | >>> namid = "namfoo" |
---|
22 | >>> text = nml.new(namid) |
---|
23 | >>> data = {"x": nml.tostring([1, 2, 3])} |
---|
24 | >>> text = nml.update(namid, text, data) |
---|
25 | >>> print text |
---|
26 | &namfoo |
---|
27 | x = 1 2 3 |
---|
28 | / |
---|
29 | <BLANKLINE> |
---|
30 | |
---|
31 | In the above snippet :func:`tostring` has been used to sanitize the input |
---|
32 | Python list. This function cleverly maps string data and numeric data to |
---|
33 | the correct Fortran syntax. |
---|
34 | |
---|
35 | However, the :func:`new` function takes care of many of the above steps automatically. |
---|
36 | Where appropriate :func:`sanitize` has been embedded to reduce the need |
---|
37 | to worry about data format problems. Take for example, |
---|
38 | |
---|
39 | >>> print nml.new("namfoo", data={"x": range(3)}) |
---|
40 | &namfoo |
---|
41 | x = 0 1 2 |
---|
42 | / |
---|
43 | <BLANKLINE> |
---|
44 | |
---|
45 | Parse existing namelist |
---|
46 | ----------------------- |
---|
47 | |
---|
48 | In order to update a namelist it is necessary to convert the namelist text into |
---|
49 | a dictionary of *key, value* pairs which can be manipulated in the usual Pythonic |
---|
50 | fashion before being piped back out to disk. |
---|
51 | |
---|
52 | In everyday usage text will be read from files, here however for illustration |
---|
53 | purposes I have hand written a namelist. |
---|
54 | |
---|
55 | >>> text = ''' |
---|
56 | ... &namfoo |
---|
57 | ... x = y ! A description of the variables |
---|
58 | ... / |
---|
59 | ... &nambar |
---|
60 | ... ln_on = .TRUE. ! A description of the variables |
---|
61 | ... / |
---|
62 | ... ''' |
---|
63 | |
---|
64 | This can be parsed by invoking the :func:`variables` command. |
---|
65 | |
---|
66 | >>> nml.variables(text) |
---|
67 | {'x': 'y', 'ln_on': '.TRUE.'} |
---|
68 | |
---|
69 | Or by using the :func:`namelists` function to split the file into sub-lists. |
---|
70 | |
---|
71 | >>> nml.namelists(text) |
---|
72 | {'namfoo': '&namfoo\\n x = y ! A description of the variables\\n/', 'nambar': '&nambar\\n ln_on = .TRUE. ! A description of the variables\\n/'} |
---|
73 | >>> sublists = nml.namelists(text) |
---|
74 | >>> print sublists["nambar"] |
---|
75 | &nambar |
---|
76 | ln_on = .TRUE. ! A description of the variables |
---|
77 | / |
---|
78 | |
---|
79 | Which can be parsed into a dictionary as before. |
---|
80 | |
---|
81 | >>> print nml.variables(sublists["nambar"]) |
---|
82 | {'ln_on': '.TRUE.'} |
---|
83 | |
---|
84 | |
---|
85 | Update/replace data |
---|
86 | ------------------- |
---|
87 | |
---|
88 | There are two ways of modifying values inside a Fortran namelist. |
---|
89 | |
---|
90 | Replace |
---|
91 | The first is to simply replace a set of variables with new values. This behaviour is accomplished |
---|
92 | via the :func:`replace` function. This approach simply overwrites existing variables. No knowledge |
---|
93 | of sub-namelist structure is required to modify a string of text. |
---|
94 | |
---|
95 | .. note:: Additional variables will not be added to a namelist via this approach |
---|
96 | |
---|
97 | Update |
---|
98 | The second is to extend the set of variables contained within a namelist. This functionality is |
---|
99 | controlled by the :func:`update` function. Here, variables which are not already specified are |
---|
100 | added using a templated namelist line. |
---|
101 | |
---|
102 | .. note:: It is essential to specify which sub-namelist is to be updated before modification takes place |
---|
103 | |
---|
104 | Pipe to/from file |
---|
105 | ----------------- |
---|
106 | |
---|
107 | As typical NEMO namelists are no larger than a few tens of kilobytes |
---|
108 | it makes sense to process namelists as single strings instead of |
---|
109 | line by line. |
---|
110 | |
---|
111 | >>> path = "foo.nml" |
---|
112 | >>> text = nml.new("namfoo") |
---|
113 | |
---|
114 | To write to a file simply invoke the writer. |
---|
115 | |
---|
116 | >>> # Write to file |
---|
117 | >>> nml.writer(path, text) |
---|
118 | |
---|
119 | To read from a file specify the path to be read. |
---|
120 | |
---|
121 | >>> # Read from file |
---|
122 | >>> text = nml.reader(path) |
---|
123 | |
---|
124 | Join multiple namelists |
---|
125 | ----------------------- |
---|
126 | |
---|
127 | Since the namelists are regular Python strings there is no need for a |
---|
128 | specific *join* function. Namelists can be combined in whatever manner |
---|
129 | is most pleasing to the eye. |
---|
130 | |
---|
131 | >>> namoff = nml.new("namoff") |
---|
132 | >>> namcl4 = nml.new("namcl4") |
---|
133 | >>> # new line join |
---|
134 | >>> print "\\n".join([namoff, namcl4]) |
---|
135 | &namoff |
---|
136 | / |
---|
137 | <BLANKLINE> |
---|
138 | &namcl4 |
---|
139 | / |
---|
140 | <BLANKLINE> |
---|
141 | |
---|
142 | >>> # Or addition |
---|
143 | >>> print namoff + namcl4 |
---|
144 | &namoff |
---|
145 | / |
---|
146 | &namcl4 |
---|
147 | / |
---|
148 | <BLANKLINE> |
---|
149 | |
---|
150 | Module functions |
---|
151 | ================ |
---|
152 | |
---|
153 | """ |
---|
154 | __version__ = "0.1.0" |
---|
155 | import re |
---|
156 | from numbers import Number |
---|
157 | |
---|
158 | def reader(path): |
---|
159 | """Reads a file into a string |
---|
160 | |
---|
161 | Reads whole file into single string. Typically, |
---|
162 | namelists are small enough to be stored in memory |
---|
163 | while updates and edits are being performed. |
---|
164 | |
---|
165 | :param path: Path to input file |
---|
166 | :returns: entire file as a single string |
---|
167 | |
---|
168 | """ |
---|
169 | with open(path, "r") as handle: |
---|
170 | text = handle.read() |
---|
171 | return text |
---|
172 | |
---|
173 | def writer(path, text): |
---|
174 | """Writes to a file from a string |
---|
175 | |
---|
176 | Handy way of piping a processed namelist into |
---|
177 | a file. |
---|
178 | |
---|
179 | :param path: Path to output file |
---|
180 | :param text: Input text to process |
---|
181 | |
---|
182 | """ |
---|
183 | with open(path, "w") as handle: |
---|
184 | handle.write(text) |
---|
185 | |
---|
186 | def update(namid, text, data, convert=True): |
---|
187 | """Extends namelist definition. |
---|
188 | |
---|
189 | Similar to replace this function alters the values |
---|
190 | of variables defined within a namelist. In addition to |
---|
191 | replacing values it also creates definitions if the |
---|
192 | variable is not found in the namelist. As such, the |
---|
193 | namelist id must be specified. |
---|
194 | |
---|
195 | :param namid: Namelist id |
---|
196 | :param text: Input text to process |
---|
197 | :param data: Dictionary of variables |
---|
198 | :keyword convert: Sanitizes input data before replacement takes place |
---|
199 | |
---|
200 | :returns: Text |
---|
201 | |
---|
202 | .. seealso:: :func:`replace` :func:`sanitize` |
---|
203 | """ |
---|
204 | sublists = namelists(text) |
---|
205 | assert namid in sublists, "Warning: invalid namid specified!" |
---|
206 | |
---|
207 | # Sanitize inputs |
---|
208 | if convert: |
---|
209 | data = sanitize(data) |
---|
210 | |
---|
211 | # Parse subsection |
---|
212 | namtext = sublists[namid] |
---|
213 | subdata = variables(namtext) |
---|
214 | subvars = subdata.keys() |
---|
215 | |
---|
216 | # Replace existing variables in namtext |
---|
217 | tmptext = replace(namtext, data) |
---|
218 | text = text.replace(namtext, tmptext) |
---|
219 | namtext = tmptext |
---|
220 | |
---|
221 | # Identify new variables |
---|
222 | vars = data.keys() |
---|
223 | newvars = list(set(vars) - set(subvars)) |
---|
224 | newvars.sort() |
---|
225 | |
---|
226 | # Append new vars to namid |
---|
227 | lines = namtext.split("\n") |
---|
228 | for v in newvars: |
---|
229 | newline = " %s = %s" % (v, data[v]) |
---|
230 | lines.insert(-1, newline) |
---|
231 | newtext = "\n".join(lines) |
---|
232 | |
---|
233 | # Replace old namtext with new namtext |
---|
234 | text = text.replace(namtext, newtext) |
---|
235 | return text |
---|
236 | |
---|
237 | def replace(text, data, convert=True): |
---|
238 | """Edits existing variables. |
---|
239 | |
---|
240 | Pattern matches and substitutes variables inside |
---|
241 | a string of text. This is independent of namid and |
---|
242 | as such is useful for modifying existing variables. |
---|
243 | To append new variables the :func:`update` function |
---|
244 | is required. |
---|
245 | |
---|
246 | >>> text = ''' |
---|
247 | ... &namobs |
---|
248 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
249 | ... / |
---|
250 | ... ''' |
---|
251 | >>> data = {"ln_sst": ".FALSE."} |
---|
252 | >>> print replace(text, data) |
---|
253 | <BLANKLINE> |
---|
254 | &namobs |
---|
255 | ln_sst = .FALSE. ! Logical switch for SST observations |
---|
256 | / |
---|
257 | <BLANKLINE> |
---|
258 | |
---|
259 | .. note :: This does not append new variables to a namelist |
---|
260 | |
---|
261 | :param text: string to process |
---|
262 | :param data: dictionary with which to modify **text** |
---|
263 | :keyword convert: Sanitizes input data before replacement takes place |
---|
264 | |
---|
265 | :returns: string with new data values |
---|
266 | |
---|
267 | .. seealso:: :func:`update`, :func:`sanitize` |
---|
268 | """ |
---|
269 | if convert: |
---|
270 | data = sanitize(data) |
---|
271 | for k, v in data.iteritems(): |
---|
272 | pat = r"(%s\s*=\s*).+?(\s*[!\n])" % (k,) |
---|
273 | repl = r"\g<1>%s\g<2>" % (v,) |
---|
274 | text = re.sub(pat, repl, text) |
---|
275 | return text |
---|
276 | |
---|
277 | def variables(text): |
---|
278 | """Retrieves dictionary of variables in text. |
---|
279 | |
---|
280 | >>> text = ''' |
---|
281 | ... &namobs |
---|
282 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
283 | ... / |
---|
284 | ... ''' |
---|
285 | >>> variables(text) |
---|
286 | {'ln_sst': '.TRUE.'} |
---|
287 | |
---|
288 | :param text: Input text to process |
---|
289 | |
---|
290 | :returns: A dictionary of variable, value pairs. |
---|
291 | |
---|
292 | """ |
---|
293 | data = {} |
---|
294 | pairs = re.findall(r"\n\s*(\w+)\s*=\s*(.+?)\s*(?=[!\n])", text) |
---|
295 | for key, value in pairs: |
---|
296 | data[key] = value |
---|
297 | return data |
---|
298 | |
---|
299 | def namelists(text): |
---|
300 | """Retrieves dictionary of namelists in text. |
---|
301 | |
---|
302 | Useful for isolating sub-namelists. |
---|
303 | |
---|
304 | >>> text = ''' |
---|
305 | ... &namobs |
---|
306 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
307 | ... / |
---|
308 | ... ''' |
---|
309 | >>> namelists(text) |
---|
310 | {'namobs': '&namobs\\n ln_sst = .TRUE. ! Logical switch for SST observations\\n/'} |
---|
311 | |
---|
312 | :param text: Input text to process |
---|
313 | |
---|
314 | :returns: A dictionary of id, text block key, value pairs |
---|
315 | |
---|
316 | """ |
---|
317 | # Boundary case |
---|
318 | if text.startswith("&"): |
---|
319 | text = "\n" + text |
---|
320 | # Regular expression |
---|
321 | results = re.findall(r"\n(&(\w+).*?\n/)", text, re.DOTALL) |
---|
322 | data = {} |
---|
323 | for content, namid in results: |
---|
324 | data[namid] = content |
---|
325 | return data |
---|
326 | |
---|
327 | def tostring(data): |
---|
328 | """Maps standard Python data to Fortran namelist format. |
---|
329 | |
---|
330 | >>> tostring([1, 2, 3]) |
---|
331 | '1 2 3' |
---|
332 | >>> tostring(["foo.nc", "bar.nc"]) |
---|
333 | "'foo.nc', 'bar.nc'" |
---|
334 | >>> tostring(True) |
---|
335 | '.TRUE.' |
---|
336 | |
---|
337 | :param data: Input Python data |
---|
338 | |
---|
339 | :returns: Namelist formatted string |
---|
340 | |
---|
341 | .. seealso:: :func:`sanitize` |
---|
342 | """ |
---|
343 | if isinstance(data, list): |
---|
344 | if all_numeric(data): |
---|
345 | delim = " " |
---|
346 | else: |
---|
347 | delim = ", " |
---|
348 | text = delim.join([convert(item) for item in data]) |
---|
349 | else: |
---|
350 | text = convert(data) |
---|
351 | return text |
---|
352 | |
---|
353 | def all_numeric(inputs): |
---|
354 | # Checks all list entries are numbers |
---|
355 | flag = True |
---|
356 | for input in inputs: |
---|
357 | if not isinstance(input, Number): |
---|
358 | flag = False |
---|
359 | break |
---|
360 | return flag |
---|
361 | |
---|
362 | def numeric(word): |
---|
363 | # Tests input string is numeric data |
---|
364 | parts = word.split(" ") |
---|
365 | try: |
---|
366 | map(float, parts) |
---|
367 | flag = True |
---|
368 | except ValueError: |
---|
369 | flag = False |
---|
370 | return flag |
---|
371 | |
---|
372 | def logical(word): |
---|
373 | # Tests input string is numeric data |
---|
374 | if word.upper() in [".FALSE.", ".TRUE."]: |
---|
375 | flag = True |
---|
376 | else: |
---|
377 | flag = False |
---|
378 | return flag |
---|
379 | |
---|
380 | def listed(word): |
---|
381 | # Tests input string is not a list |
---|
382 | if ("," in word) or (" " in word): |
---|
383 | flag = True |
---|
384 | else: |
---|
385 | flag = False |
---|
386 | return flag |
---|
387 | |
---|
388 | def quote(word): |
---|
389 | word = str(word) |
---|
390 | if not quoted(word): |
---|
391 | word = "'%s'" % (word,) |
---|
392 | return word |
---|
393 | |
---|
394 | def convert(word): |
---|
395 | # Conversion function |
---|
396 | if isinstance(word, str): |
---|
397 | if (quoted(word) or numeric(word) |
---|
398 | or logical(word) or listed(word)): |
---|
399 | result = "%s" % (word,) |
---|
400 | else: |
---|
401 | result = quote(word) |
---|
402 | elif isinstance(word, bool): |
---|
403 | if word: |
---|
404 | result = ".TRUE." |
---|
405 | else: |
---|
406 | result = ".FALSE." |
---|
407 | else: |
---|
408 | result = str(word) |
---|
409 | return result |
---|
410 | |
---|
411 | def quoted(word): |
---|
412 | # Checks if string begins/ends with quotation marks |
---|
413 | if (word.startswith("'") and word.endswith("'")): |
---|
414 | flag = True |
---|
415 | elif (word.startswith('"') and word.endswith('"')): |
---|
416 | flag = True |
---|
417 | else: |
---|
418 | flag = False |
---|
419 | return flag |
---|
420 | |
---|
421 | def same_type(data): |
---|
422 | # True if all entries are the same type |
---|
423 | types = map(type, data) |
---|
424 | if len(set(types)) == 1: |
---|
425 | flag = True |
---|
426 | else: |
---|
427 | flag = False |
---|
428 | return flag |
---|
429 | |
---|
430 | def sanitize(data): |
---|
431 | """Converts dictionary values into Fortran namelist format. |
---|
432 | |
---|
433 | This is a more typical way to prepare data for inclusion in |
---|
434 | a Fortran namelist. Instead of manually applying :func:`tostring` |
---|
435 | to every element of the input data, **sanitize** fixes the entire |
---|
436 | data set. |
---|
437 | |
---|
438 | >>> sanitize({"x": True}) |
---|
439 | {'x': '.TRUE.'} |
---|
440 | >>> |
---|
441 | |
---|
442 | :param data: Dictionary to convert |
---|
443 | |
---|
444 | :returns: Dictionary whose values are in Fortran namelist format |
---|
445 | |
---|
446 | .. seealso:: :func:`tostring` |
---|
447 | """ |
---|
448 | replacements = [(k, tostring(v)) for k, v in data.items()] |
---|
449 | data.update(replacements) |
---|
450 | return data |
---|
451 | |
---|
452 | def new(namid, data=None, convert=True): |
---|
453 | """Creates a new Fortran namelist |
---|
454 | |
---|
455 | >>> new("namobs") |
---|
456 | '&namobs\\n/\\n' |
---|
457 | >>> print new("namobs") |
---|
458 | &namobs |
---|
459 | / |
---|
460 | <BLANKLINE> |
---|
461 | |
---|
462 | :param namid: Name for the new namelist |
---|
463 | |
---|
464 | :keyword data: Specifies an initial dictionary with which to |
---|
465 | populate the namelist |
---|
466 | :type data: dict |
---|
467 | :keyword convert: Sanitizes input data before replacement takes place |
---|
468 | |
---|
469 | :returns: string representation of a Fortran namelist |
---|
470 | """ |
---|
471 | text = "&{namid}\n/\n".format(namid=namid) |
---|
472 | if data is not None: |
---|
473 | text = update(namid, text, data, convert=convert) |
---|
474 | return text |
---|
475 | |
---|
476 | if __name__ == '__main__': |
---|
477 | import doctest |
---|
478 | doctest.testmod() |
---|
479 | |
---|