1 | """Collection of Fortran 90 namelist helper functions. |
---|
2 | |
---|
3 | A common way to interface with a Fortran executable is via |
---|
4 | an input file called a namelist. This module defines |
---|
5 | functions which simplify the process of updating and |
---|
6 | extending namelist data. |
---|
7 | |
---|
8 | .. note:: This module is especially lightweight and follows the |
---|
9 | batteries included philosophy. As such, only standard |
---|
10 | library modules are required to use this code. |
---|
11 | |
---|
12 | Walkthrough |
---|
13 | =========== |
---|
14 | |
---|
15 | New namelist |
---|
16 | ------------ |
---|
17 | |
---|
18 | A typical usage is to create and update a Fortran namelist on the fly. |
---|
19 | |
---|
20 | >>> import nml |
---|
21 | >>> namid = "namfoo" |
---|
22 | >>> text = nml.new(namid) |
---|
23 | >>> data = {"x": nml.tostring([1, 2, 3])} |
---|
24 | >>> text = nml.update(namid, text, data) |
---|
25 | >>> print text |
---|
26 | <BLANKLINE> |
---|
27 | &namfoo |
---|
28 | x = 1 2 3 |
---|
29 | / |
---|
30 | |
---|
31 | In the above snippet :func:`tostring` has been used to sanitize the input |
---|
32 | Python list. This function cleverly maps string data and numeric data to |
---|
33 | the correct Fortran syntax. |
---|
34 | |
---|
35 | However, the :func:`new` function takes care of many of the above steps automatically. |
---|
36 | Where appropriate :func:`sanitize` has been embedded to reduce the need |
---|
37 | to worry about data format problems. Take for example, |
---|
38 | |
---|
39 | >>> print nml.new("namfoo", data={"x": range(3)}) |
---|
40 | <BLANKLINE> |
---|
41 | &namfoo |
---|
42 | x = 0 1 2 |
---|
43 | / |
---|
44 | |
---|
45 | Parse existing namelist |
---|
46 | ----------------------- |
---|
47 | |
---|
48 | In order to update a namelist it is necessary to convert the namelist text into |
---|
49 | a dictionary of *key, value* pairs which can be manipulated in the usual Pythonic |
---|
50 | fashion before being piped back out to disk. |
---|
51 | |
---|
52 | In everyday usage text will be read from files, here however for illustration |
---|
53 | purposes I have hand written a namelist. |
---|
54 | |
---|
55 | >>> text = ''' |
---|
56 | ... &namfoo |
---|
57 | ... x = y ! A description of the variables |
---|
58 | ... / |
---|
59 | ... &nambar |
---|
60 | ... ln_on = .TRUE. ! A description of the variables |
---|
61 | ... / |
---|
62 | ... ''' |
---|
63 | |
---|
64 | This can be parsed by invoking the :func:`variables` command. |
---|
65 | |
---|
66 | >>> nml.variables(text) |
---|
67 | {'x': 'y', 'ln_on': '.TRUE.'} |
---|
68 | |
---|
69 | Or by using the :func:`namelists` function to split the file into sub-lists. |
---|
70 | |
---|
71 | >>> nml.namelists(text) |
---|
72 | {'namfoo': '&namfoo\\n x = y ! A description of the variables\\n/', 'nambar': '&nambar\\n ln_on = .TRUE. ! A description of the variables\\n/'} |
---|
73 | >>> sublists = nml.namelists(text) |
---|
74 | >>> print sublists["nambar"] |
---|
75 | &nambar |
---|
76 | ln_on = .TRUE. ! A description of the variables |
---|
77 | / |
---|
78 | |
---|
79 | Which can be parsed into a dictionary as before. |
---|
80 | |
---|
81 | >>> print nml.variables(sublists["nambar"]) |
---|
82 | {'ln_on': '.TRUE.'} |
---|
83 | |
---|
84 | |
---|
85 | Update/replace data |
---|
86 | ------------------- |
---|
87 | |
---|
88 | There are two ways of modifying values inside a Fortran namelist. |
---|
89 | |
---|
90 | Replace |
---|
91 | The first is to simply replace a set of variables with new values. This behaviour is accomplished |
---|
92 | via the :func:`replace` function. This approach simply overwrites existing variables. No knowledge |
---|
93 | of sub-namelist structure is required to modify a string of text. |
---|
94 | |
---|
95 | .. note:: Additional variables will not be added to a namelist via this approach |
---|
96 | |
---|
97 | Update |
---|
98 | The second is to extend the set of variables contained within a namelist. This functionality is |
---|
99 | controlled by the :func:`update` function. Here, variables which are not already specified are |
---|
100 | added using a templated namelist line. |
---|
101 | |
---|
102 | .. note:: It is essential to specify which sub-namelist is to be updated before modification takes place |
---|
103 | |
---|
104 | Pipe to/from file |
---|
105 | ----------------- |
---|
106 | |
---|
107 | As typical NEMO namelists are no larger than a few tens of kilobytes |
---|
108 | it makes sense to process namelists as single strings instead of |
---|
109 | line by line. |
---|
110 | |
---|
111 | >>> path = "foo.nml" |
---|
112 | >>> text = nml.new("namfoo") |
---|
113 | |
---|
114 | To write to a file simply invoke the writer. |
---|
115 | |
---|
116 | >>> # Write to file |
---|
117 | >>> nml.writer(path, text) |
---|
118 | |
---|
119 | To read from a file specify the path to be read. |
---|
120 | |
---|
121 | >>> # Read from file |
---|
122 | >>> text = nml.reader(path) |
---|
123 | |
---|
124 | Join multiple namelists |
---|
125 | ----------------------- |
---|
126 | |
---|
127 | Since the namelists are regular Python strings there is no need for a |
---|
128 | specific *join* function. Namelists can be combined in whatever manner |
---|
129 | is most pleasing to the eye. |
---|
130 | |
---|
131 | >>> namoff = nml.new("namoff") |
---|
132 | >>> namcl4 = nml.new("namcl4") |
---|
133 | >>> # new line join |
---|
134 | >>> print "\\n".join([namoff, namcl4]) |
---|
135 | <BLANKLINE> |
---|
136 | &namoff |
---|
137 | / |
---|
138 | <BLANKLINE> |
---|
139 | &namcl4 |
---|
140 | / |
---|
141 | |
---|
142 | >>> # Or addition |
---|
143 | >>> print namoff + namcl4 |
---|
144 | <BLANKLINE> |
---|
145 | &namoff |
---|
146 | / |
---|
147 | &namcl4 |
---|
148 | / |
---|
149 | |
---|
150 | Module functions |
---|
151 | ================ |
---|
152 | |
---|
153 | """ |
---|
154 | __version__ = "0.1.0" |
---|
155 | |
---|
156 | import re |
---|
157 | |
---|
158 | def reader(path): |
---|
159 | """Reads a file into a string |
---|
160 | |
---|
161 | Reads whole file into single string. Typically, |
---|
162 | namelists are small enough to be stored in memory |
---|
163 | while updates and edits are being performed. |
---|
164 | |
---|
165 | :param path: Path to input file |
---|
166 | :returns: entire file as a single string |
---|
167 | |
---|
168 | """ |
---|
169 | with open(path, "r") as handle: |
---|
170 | text = handle.read() |
---|
171 | return text |
---|
172 | |
---|
173 | def writer(path, text): |
---|
174 | """Writes to a file from a string |
---|
175 | |
---|
176 | Handy way of piping a processed namelist into |
---|
177 | a file. |
---|
178 | |
---|
179 | :param path: Path to output file |
---|
180 | :param text: Input text to process |
---|
181 | |
---|
182 | """ |
---|
183 | with open(path, "w") as handle: |
---|
184 | handle.write(text) |
---|
185 | |
---|
186 | def update(namid, text, data, convert=True): |
---|
187 | """Extends namelist definition. |
---|
188 | |
---|
189 | Similar to replace this function alters the values |
---|
190 | of variables defined within a namelist. In addition to |
---|
191 | replacing values it also creates definitions if the |
---|
192 | variable is not found in the namelist. As such, the |
---|
193 | namelist id must be specified. |
---|
194 | |
---|
195 | :param namid: Namelist id |
---|
196 | :param text: Input text to process |
---|
197 | :param data: Dictionary of variables |
---|
198 | :keyword convert: Sanitizes input data before replacement takes place |
---|
199 | |
---|
200 | :returns: Text |
---|
201 | |
---|
202 | .. seealso:: :func:`replace` :func:`sanitize` |
---|
203 | """ |
---|
204 | sublists = namelists(text) |
---|
205 | assert namid in sublists, "Warning: invalid namid specified!" |
---|
206 | |
---|
207 | # Sanitize inputs |
---|
208 | if convert: |
---|
209 | data = sanitize(data) |
---|
210 | |
---|
211 | # Parse subsection |
---|
212 | namtext = sublists[namid] |
---|
213 | subdata = variables(namtext) |
---|
214 | subvars = subdata.keys() |
---|
215 | |
---|
216 | # Replace existing variables in namtext |
---|
217 | tmptext = replace(namtext, data) |
---|
218 | text = text.replace(namtext, tmptext) |
---|
219 | namtext = tmptext |
---|
220 | |
---|
221 | # Identify new variables |
---|
222 | vars = data.keys() |
---|
223 | newvars = list(set(vars) - set(subvars)) |
---|
224 | newvars.sort() |
---|
225 | |
---|
226 | # Append new vars to namid |
---|
227 | lines = namtext.split("\n") |
---|
228 | for v in newvars: |
---|
229 | newline = " %s = %s" % (v, data[v]) |
---|
230 | lines.insert(-1, newline) |
---|
231 | newtext = "\n".join(lines) |
---|
232 | |
---|
233 | # Replace old namtext with new namtext |
---|
234 | text = text.replace(namtext, newtext) |
---|
235 | return text |
---|
236 | |
---|
237 | def replace(text, data, convert=True): |
---|
238 | """Edits existing variables. |
---|
239 | |
---|
240 | Pattern matches and substitutes variables inside |
---|
241 | a string of text. This is independent of namid and |
---|
242 | as such is useful for modifying existing variables. |
---|
243 | To append new variables the :func:`update` function |
---|
244 | is required. |
---|
245 | |
---|
246 | >>> text = ''' |
---|
247 | ... &namobs |
---|
248 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
249 | ... / |
---|
250 | ... ''' |
---|
251 | >>> data = {"ln_sst": ".FALSE."} |
---|
252 | >>> print replace(text, data) |
---|
253 | <BLANKLINE> |
---|
254 | &namobs |
---|
255 | ln_sst = .FALSE. ! Logical switch for SST observations |
---|
256 | / |
---|
257 | <BLANKLINE> |
---|
258 | |
---|
259 | .. note :: This does not append new variables to a namelist |
---|
260 | |
---|
261 | :param text: string to process |
---|
262 | :param data: dictionary with which to modify **text** |
---|
263 | :keyword convert: Sanitizes input data before replacement takes place |
---|
264 | |
---|
265 | :returns: string with new data values |
---|
266 | |
---|
267 | .. seealso:: :func:`update`, :func:`sanitize` |
---|
268 | """ |
---|
269 | if convert: |
---|
270 | data = sanitize(data) |
---|
271 | for k, v in data.iteritems(): |
---|
272 | pat = r"(%s\s+=\s+).+?(\s*[!\n])" % (k,) |
---|
273 | repl = r"\g<1>%s\g<2>" % (v,) |
---|
274 | text = re.sub(pat, repl, text) |
---|
275 | return text |
---|
276 | |
---|
277 | def variables(text): |
---|
278 | """Retrieves dictionary of variables in text. |
---|
279 | |
---|
280 | >>> text = ''' |
---|
281 | ... &namobs |
---|
282 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
283 | ... / |
---|
284 | ... ''' |
---|
285 | >>> variables(text) |
---|
286 | {'ln_sst': '.TRUE.'} |
---|
287 | |
---|
288 | :param text: Input text to process |
---|
289 | |
---|
290 | :returns: A dictionary of variable, value pairs. |
---|
291 | |
---|
292 | """ |
---|
293 | data = {} |
---|
294 | pairs = re.findall(r"\n\s*(\w+)\s*=\s*(.+?)\s*(?=[!\n])", text) |
---|
295 | for key, value in pairs: |
---|
296 | data[key] = value |
---|
297 | return data |
---|
298 | |
---|
299 | def namelists(text): |
---|
300 | """Retrieves dictionary of namelists in text. |
---|
301 | |
---|
302 | Useful for isolating sub-namelists. |
---|
303 | |
---|
304 | >>> text = ''' |
---|
305 | ... &namobs |
---|
306 | ... ln_sst = .TRUE. ! Logical switch for SST observations |
---|
307 | ... / |
---|
308 | ... ''' |
---|
309 | >>> namelists(text) |
---|
310 | {'namobs': '&namobs\\n ln_sst = .TRUE. ! Logical switch for SST observations\\n/'} |
---|
311 | |
---|
312 | :param text: Input text to process |
---|
313 | |
---|
314 | :returns: A dictionary of id, text block key, value pairs |
---|
315 | |
---|
316 | """ |
---|
317 | # Boundary case |
---|
318 | if text.startswith("&"): |
---|
319 | text = "\n" + text |
---|
320 | # Regular expression |
---|
321 | results = re.findall(r"\n(&(\w+).*?\n/)", text, re.DOTALL) |
---|
322 | data = {} |
---|
323 | for content, namid in results: |
---|
324 | data[namid] = content |
---|
325 | return data |
---|
326 | |
---|
327 | def tostring(data): |
---|
328 | """Maps standard Python data to Fortran namelist format. |
---|
329 | |
---|
330 | >>> tostring([1, 2, 3]) |
---|
331 | '1 2 3' |
---|
332 | >>> tostring(["foo.nc", "bar.nc"]) |
---|
333 | "'foo.nc' 'bar.nc'" |
---|
334 | >>> tostring(True) |
---|
335 | '.TRUE.' |
---|
336 | |
---|
337 | :param data: Input Python data |
---|
338 | |
---|
339 | :returns: Namelist formatted string |
---|
340 | |
---|
341 | .. seealso:: :func:`sanitize` |
---|
342 | """ |
---|
343 | if isinstance(data, list): |
---|
344 | # Conversion function |
---|
345 | def convert(word): |
---|
346 | if isinstance(word, str): |
---|
347 | result = "'%s'" % (word,) |
---|
348 | else: |
---|
349 | result = str(word) |
---|
350 | return result |
---|
351 | text = " ".join([convert(item) for item in data]) |
---|
352 | elif isinstance(data, bool): |
---|
353 | if data: |
---|
354 | text = ".TRUE." |
---|
355 | else: |
---|
356 | text = ".FALSE." |
---|
357 | else: |
---|
358 | text = str(data) |
---|
359 | return text |
---|
360 | |
---|
361 | def sanitize(data): |
---|
362 | """Converts dictionary values into Fortran namelist format. |
---|
363 | |
---|
364 | This is a more typical way to prepare data for inclusion in |
---|
365 | a Fortran namelist. Instead of manually applying :func:`tostring` |
---|
366 | to every element of the input data, **sanitize** fixes the entire |
---|
367 | data set. |
---|
368 | |
---|
369 | >>> sanitize({"x": True}) |
---|
370 | {'x': '.TRUE.'} |
---|
371 | >>> |
---|
372 | |
---|
373 | :param data: Dictionary to convert |
---|
374 | |
---|
375 | :returns: Dictionary whose values are in Fortran namelist format |
---|
376 | |
---|
377 | .. seealso:: :func:`tostring` |
---|
378 | """ |
---|
379 | replacements = [(k, tostring(v)) for k, v in data.items()] |
---|
380 | data.update(replacements) |
---|
381 | return data |
---|
382 | |
---|
383 | def new(namid, data=None, convert=True): |
---|
384 | """Creates a new Fortran namelist |
---|
385 | |
---|
386 | >>> new("namobs") |
---|
387 | '\\n&namobs\\n/' |
---|
388 | >>> print new("namobs") |
---|
389 | <BLANKLINE> |
---|
390 | &namobs |
---|
391 | / |
---|
392 | |
---|
393 | :param namid: Name for the new namelist |
---|
394 | |
---|
395 | :keyword data: Specifies an initial dictionary with which to |
---|
396 | populate the namelist |
---|
397 | :type data: dict |
---|
398 | :keyword convert: Sanitizes input data before replacement takes place |
---|
399 | |
---|
400 | :returns: string representation of a Fortran namelist |
---|
401 | """ |
---|
402 | text = "\n&{namid}\n/".format(namid=namid) |
---|
403 | if data is not None: |
---|
404 | text = update(namid, text, data, convert=convert) |
---|
405 | return text |
---|
406 | |
---|
407 | if __name__ == '__main__': |
---|
408 | import doctest |
---|
409 | doctest.testmod() |
---|
410 | |
---|